Databricks Certified Associate Developer for Apache Spark 3

Databricks Certified Associate Developer for Apache Spark 3.0 Program

A Step by Step Hands-on Guide to prepare for one of the most challenging certifications, the Databricks Certified Associate Developer for Apache Spark 3.0.

Enroll Now

Course curriculum

1. Introduction to Databricks Certified Associate for Spark Developer Course
2. Sign up for Databricks Academy Website
3. Get Details related to Databricks Certified Associate exam for Spark Developers
4. Overview of Databricks Certified Associate for Apache Spark Curriculum
5. Resources to prepare for Databricks Certified Associate Spark Developer Exam
6. Exam Details for Databricks Certified Associate Developer for Apache Spark
7. Registering for Databricks Certified Associate Developer for Apache Spark
1. Sign up for Azure Portal
2. Setup Databricks Platform using Azure
3. Prerequisites for the Databricks Spark Developer Certification
4. Create Single Node Cluster to explore Spark APIs
5. Getting Started with Databricks Notebooks
6. Setup Databricks Certification Course Material
7. Quick Tour of Course Material using Databricks Notebooks
8. Install and Configure Databricks CLI
9. Interacting with File System using CLI
10. Setup Retail Datasets using Databricks CLI
11. Validate Data Sets using Databricks Notebooks
1. Create Spark Dataframes using Python Collections and Pandas Dataframes - Introduction
2. Create Single Column Spark Dataframe using List
3. Create Multi Column Spark Dataframe using List
4. Overview of Spark Row
5. Convert List of Lists into Spark Dataframe using Row
6. Convert List of Tuples into Spark Dataframe using Row
7. Convert List of Dicts into Spark Dataframe using Row
8. Overview of Basic Data Types in Spark
9. Specifying Schema for Spark Dataframe using String
10. Specifying Schema for Spark Dataframe using List
11. Specifying Schema using Spark Types
12. Create Spark Dataframe using Pandas Dataframe
13. Overview of Special Data Types in Spark
14. Array Type Columns in Spark Dataframes
15. Map Type Columns in Spark Dataframes
16. Struct Type Columns in Spark Dataframes
1. Selecting and Renaming Columns in Spark Data Frames - Introduction
2. Creating Spark Data Frame to Select and Rename Columns
3. Overview of Narrow and Wide Transformations
4. Overview of Select on Spark Data Frame
5. Overview of selectExpr on Spark Data Frame
6. Referring Columns using Spark Data Frame Names
7. Understanding col function in Spark
8. Invoking Functions using Spark Column Objects
9. Understanding lit function in Spark
10. Overview of Renaming Spark Data Frame Columns or Expressions
11. Naming derived columns using withColumn
12. Renaming Columns using withColumnRenamed
13. Renaming Spark Data Frame columns or expressions using alias
14. Renaming and Reordering multiple Spark Data Frame Columns
1. Manipulating Columns in Spark Data Frames - Introduction
2. Predefined Functions using Spark Data Frame APIs
3. Create Dummy Spark Data Frames
4. Categories Of Functions to Manipulate Columns in Spark Data Frames
5. Getting Help on Spark Functions
6. Special Functions col and lit using Spark
7. Common String Manipulation Functions on Spark Data Frames
8. Extracting Strings using substring from Spark Data Frame Columns
9. Extracting Strings using split from Spark Data Frame Columns
10. Padding Characters around strings in Spark Data Frame Columns
11. Trimming Characters from strings in Spark Data Frame Columns
12. Date and Time Manipulation Functions using Spark Data Frames
13. Date and Time Arithmetic using Spark Data Frames
14. Using date and time trunc functions on Spark Data Frames
15. Date and Time Extract Functions on Spark Data Frames
16. Using to_date and to_timestamp on Spark Data Frames
17. Using date_format Function on Spark Data Frames
18. Dealing with Unix Timestamp in Spark Data Frames
19. Dealing with nulls in Spark Data Frames
1. Filtering Data from Spark Data Frames - Introduction
2. Creating Spark Data Frame for Filtering
3. Overview of Filter or Where Function on Spark Data Frame
4. Overview of Conditions and Operators related to Spark Data Frames
5. Filter using Equal Condition on Spark Data Frames
6. Filter using Not Equal Condition on Spark Data Frames
7. Filter using Between Operator on Spark Data Frames
8. Dealing with Null Values while Filtering Data in Spark Data Frames
9. Overview of Boolean Operations
10. Boolean OR on same column of Spark Data Frame and IN Operator
11. Filtering with Greater Than and Less Than on Spark Data Frames
12. Boolean AND Condition on Spark Data Frames
13. Boolean OR on different columns of a Spark Data Frame
1. Dropping Columns from Spark Data Frames - Introduction
2. Creating Spark Data Frame for Dropping Columns
3. Overview of Spark Data Frame drop function
4. Dropping a Single Column from a Spark Data Frame
5. Dropping Multiple Columns from a Spark Data Frame
6. Dropping List of Columns from a Spark Data Frame
7. Dropping Duplicate Records from Spark Data Frames
8. Dropping Null based Records from Spark Data Frames
1. Sorting Data in Spark Data Frames - Introduction
2. Creating Spark Data Frame for Sorting the Data
3. Overview of Sorting a Spark Data Frame
4. Sort Spark Data Frame in Ascending Order by a given column
5. Sort Spark Data Frame in Descending Order by a given column
6. Dealing with Nulls while sorting Spark Data Frame
7. Composite Sorting of a Data Frame
8. Prioritized Sorting of a Spark Data Frame
1. Performing Aggregations on Spark Data Frames - Introduction
2. Validate Data Sets for Aggregations using Spark
3. Common Spark Aggregate Functions
4. Total Aggregations on a Spark Data Frame
5. Getting Count of a Spark Data Frame
6. Overview of groupBy on Spark Data Frame
7. Perform Grouped Aggregations using direct functions on a Spark Data Frame
8. Perform Grouped Aggregations using agg on a Spark Data Frame
1. Joining Spark Data Frames - Introduction
2. Setup Data Sets to perform joins
3. Overview of Joins using Spark Data Frames
4. Define Aliases for Spark Data Frames
5. Performing Inner Join on Spark Data Frames
6. Performing Outer Join using left between Spark Data Frames
7. Performing Outer Join using right between Spark Data Frames
8. Difference between Left Outer Join and Right Outer Join
9. Performing Full Outer Join between Spark Dataframes
10. Overview of Broadcast Join in Spark
11. Performing Cross Join using Spark Data Frames
1. Reading Data from Spark Data Frames into Files - Introduction
2. Validate Data Sets for Reading from Files using Spark APIs
3. Convert JSON Data to Parquet using Spark APIs
4. Convert Comma Separated Files to Pipe Separated Files using Spark
5. Overview of Reading Data Files into Spark Data Frames
6. Steps to follow to read data from files into Spark Data Frame
7. Reading Data from CSV files into Spark Data Frame
8. Specifying Schema while reading CSV data into Data Frame
9. Using toDF and inferSchema using CSV to create Spark Data Frame
10. Specifying Delimiter while using CSV to create Spark Data Frame
11. Using Options while reading CSV Files into Spark Data Frame
12. Reading JSON Files into Spark Data Frame
13. Specifying Schema while reading JSON Files into Data Frame
14. Side effects of inferring schema while creating Spark Data Frame
15. Reading Parquet Files into Spark Data Frame
16. Specifying Schema while reading Parquet Files into Data Frame
1. Writing Data from Spark Data Frames into Files - Introduction
2. Validate Data Sets for Writing into Files using Spark APIs
3. Overview of Writing Spark Data Frames into Files
4. Steps to follow to write Spark Data Frames into Files
5. Writing Spark Data Frames into CSV files
6. Specifying Header while writing Spark Data Frame into CSV files
7. Using Compression while writing Spark Data Frame into CSV Files
8. Specifying Delimiter while writing Spark Data Frame into CSV Files
9. Using Options while writing Spark Data Frame into CSV Files
10. Writing Spark Data Frames into JSON Files
11. Compression while writing Spark Data Frames into JSON Files
12. Writing Spark Data Frames into Parquet Files
13. Compression while writing Spark Data Frames into Parquet Files
14. Different Modes to write Spark Data Frame into Files
15. Coalesce and Repartitioning of Spark Data Frames
1. Partitioning Spark Data Frames - Introduction
2. Overview of Partitioning Data Frames
3. Partition Spark Data Frame By Single Column
4. Partition Spark Data Frame By Multiple Columns
5. Setup Data Set for Partition Pruning
6. Reading Data into Spark Data Frames using Partition Pruning
1. Overview of Spark User Defined Functions
2. Registering Spark User Defined Functions
3. Using Spark UDFs as part of Data Frame APIs
4. Using Spark UDFs as part of Spark SQL
5. Cleansing Data using Spark UDFs
1. Setup Multinode Spark Cluster using Databricks Platform
2. Review important Databricks Spark User Interfaces
3. Overview of Spark Cores and Slots
4. Submit Basic Application to understand tasks
5. Run Spark Application to understand the Execution Life Cycle
6. Review Properties related to Spark Adaptive Execution
7. Disabling Adaptive Execution and Running Spark Application
8. Review Details of Spark Application with out Adaptive Execution
9. Run Spark Application with Adaptive Execution
10. Review Spark Adaptive Query Execution Properties
1. Overview of the Material Provided for Databricks Apache Spark Exam
2. Accessing the Mock Test
3. Understanding Mock Test for Databricks Certified Associate Spark Exam

About this course

170 lessons
14.5 hours of video content

Databricks Certified Associate Developer for Apache Spark 3.0 Program

Course curriculum

Getting Started with Databricks Certified Associate Developer for Apache Spark

Setup Databricks Environment using Azure

Create Spark Dataframes using Python Collections and Pandas Dataframes

Selecting and Renaming Columns in Spark Data Frames

Manipulating Columns in Spark Data Frames

Filtering Data from Spark Data Frames

Dropping Columns from Spark Data Frames

Sorting Data in Spark Data Frames

Performing Aggregations on Spark Data Frames

Joining Spark Data Frames

Reading Data from Spark Data Frames into Files

Writing Data from Files into Spark Data Frames

Partitioning Spark Data Frames

Working with Spark SQL Functions

Spark Architecture

99 Tips and Strategies to take the Databricks Certified Associate Developer for Apache Spark Exam

About this course