Data Engineering Course – Power BI, Azure, SQL, SSIS, Python

For Automation Masters

Workflow Automation + Integrated Systems
The Future of Process Automation.

Gain the skills to optimize both on-prem and cloud technologies for reporting and workflow automation. Ideal for professionals who want to seamlessly blend legacy systems with modern cloud solutions for smarter, faster processes.

Tools You'll Learn

Visualization Tool

Power BI

Application Development Tools

Power Apps

Automation Tools

Power Automate, VBA, Python, Azure Logic Apps, Azure Data Factory Triggers

Data Modeling Tools

Advanced DAX Functions, Azure Analysis Services, Power BI Dataset, SSAS

Data Transformation Tools

Power Query, Azure Data Factory, Data Bricks, SSIS

Data Warehousing & Querying Tools

SQL, Azure SQL Database, Azure Synapse Analytics

Cloud Data Integration & Management

Azure Data Lake, Azure Blob Storage, Power BI Service

In just 6 weeks, you'll be able to:

Become a Hybrid BI & Data Engineering Expert – Master both on-premise (SQL Server, SSIS, SSAS, VBA) and cloud-based (Azure, Power BI, Power Automate) tools.

Build & Automate Scalable Data Pipelines – Extract, transform, and load (ETL) data using SSIS, Azure Data Factory, Data Bricks, and Python.

Design Enterprise-Grade BI Solutions – Develop interactive Power BI dashboards, optimize performance with Advanced DAX, and manage Analysis Services models.

Enhance Data Processing & Integration – Use SQL Querying, SQL Programming, and Azure Synapse Analytics to store, analyze, and manage large-scale data.

Streamline Business Automation – Leverage Power Automate, Azure Logic Apps, and VBA Web Scraping to automate workflows and data movements.

Bridge Traditional & Modern BI Tools – Integrate Excel, VBA, SQL, and Power BI for seamless data connectivity and enhanced reporting.

In just 6 weeks, you'll be able to:

Become a Hybrid BI & Data Engineering Expert – Master both on-premise (SQL Server, SSIS, SSAS, VBA) and cloud-based (Azure, Power BI, Power Automate) tools.

Build & Automate Scalable Data Pipelines – Extract, transform, and load (ETL) data using SSIS, Azure Data Factory, Data Bricks, and Python.

Design Enterprise-Grade BI Solutions – Develop interactive Power BI dashboards, optimize performance with Advanced DAX, and manage Analysis Services models.

Enhance Data Processing & Integration – Use SQL Querying, SQL Programming, and Azure Synapse Analytics to store, analyze, and manage large-scale data.

Streamline Business Automation – Leverage Power Automate, Azure Logic Apps, and VBA Web Scraping to automate workflows and data movements.

Bridge Traditional & Modern BI Tools – Integrate Excel, VBA, SQL, and Power BI for seamless data connectivity and enhanced reporting.

Ideal For:

Techno-business professionals

managing hybrid infrastructures for BI reporting and automation.

BI Professionals

who need expertise in both on-premise and cloud environments.

Digital Transformation Leaders

overseeing hybrid BI strategies across organizations.

Non-Technical Professionals

looking to bridge the gap between on-premise and cloud solutions.

Enterprise Architects

designing solutions that combine on-premise and cloud technologies.

A snapshot of what you'll be learning in 6-weeks.

Course Syllabus Overview

Advanced Charting Techniques

The Science of Chart Choice

Decode the intent behind visuals: comparison, distribution, trend, or composition
Know when to use Clustered vs Stacked vs 100% Column Charts
Choose the right axis, scale, and layout based on your message
Avoid misleading visuals and cluttered dashboards

Smart Charting Techniques

Master column, bar, line, area, and combo charts with use-case depth
Conditional charts to highlight exceptions, variances, or risk points
Use formulas to drive attention: Top N, outliers, or KPI status

Data Storytelling Framework

Learn the 3-layer structure: Data → Insight → Impactful Visual
Build visuals that answer business questions, not just decorate reports
Turn dry tables into executive-ready visuals

Interactive & Dynamic Charting

Drop-down-driven visuals for user-led exploration
Pivot-linked visuals for real-time insights
Use slicers and timelines like a pro

Design for Interpretation

Clean layout, minimal ink, maximum clarity
Design tricks: focus colors, white space, custom labels, layering
Chart templates for consistency across reports

Quick Excel brush-up

Fundamentals Of Functions & Formulas Quick techniques to work with formulas & functions DATE AND TIME FUNCTIONS For data and time calculations

Today, Now
Day, Month, Year
Date, DateDif, DateAdd
EOMonth, Weekday
Workdays, NetWorkdays, Edate

TEXT FUNCTIONS For data transformations

Upper, Lower, Proper
Left, Mid, Right
Trim, Len
Concatenate
Find, Substitute

LOGICAL FUNCTIONS For conditional statement development

Nested If ( And Conditions , Or Conditions )
Alternative Solutions for Complex IF Conditions to make work simple
And, Or, Not

MATHEMATICAL FUNCTIONS For fundamental analysis

Sum, Count, CountA, Average, AverageA, Max & Min
SumIf, SumIfs
CountIf, CountIfs
AverageIf, AverageIfs
MaxIfs, MinIFs

LOOKUP FUNCTIONS For joins with multiple tables

Vlookup / HLookup
Match
Dynamic Two Way Lookup
Creating Smooth User Interface Using Lookup
Offset
Index
Dynamic Worksheet linking using Indirect

ERROR HANDLING FUNCTIONS For handling errors in functions

VBA

Understanding VBA Programming Environment

Overview

Project
Project Explorer
Menubar
Toolbar
Code Window
Module
UserForm

Programming Fundamentals Using VBA

Procedures
User Defined Functions
InputBox Function
MsgBox Function
Variables
Constants
Data Types
Working with Logical Operators
IF Statement
Nested IF Statement
IF with And Or Scenarios
Select Case
VBA Functions
Problem Solving Techniques Required for Programming
Saving Macro Enabled Files
For Loops
Do Loops
While...Wend Statement
Protecting Code
Error Handling

Running Code

Run mode as a developer
Running Code as end user
Running Code to debug procedures & Functions

Connecting to MS-SQL via VBA

Working with MS-SQL Applications

Creating a Database
Understanding Tables and Creating Tables
Querying Data
Filtering Data
Grouping Data
Ordering Data
Column Aliases
Table Aliases
Retrieving data with VBA
Inserting, Updating and Deleting Data with VBA

Web Scrapping With VBA

Web Scrapping with VBA

Understanding & Installing Chrome Driver
Understanding HTML Tags
Understanding IDs, Class & Elements
Understanding basic Javascripts
Reading & Updating Websites using Element IDs, Class & Names
Reading Dynamic Elements in websites using VBA
Auto Filling Complex WebForms with VBA
Performing Web based actions using VBA

System Integration Using RPA

Integrating with other web based applications using API

Understanding APIs
Understanding JSON File Types and XML Files
Creating API calls with VBA
Reading, updating & deleting data with API
Working with Google APIs, Microsoft APIs, SalesForce APIs, etc

Power BI

Building Blocks of Power BI

Visualizations
Datasets
Reports
Dashboards
Tiles

Building Your First Power BI Report

Connect to Data Sources in Power BI Desktop
Clean and Transform Your Data With the Query Editor
Create a report in Power BI Desktop
Publish the report in the Power BI service

Data Modelling with Power BI

Fundamentals of Modelling
How to Manage Your Data Relationships
Create Calculated Columns
Optimizing Data Models for Better Visuals
Create measures and work with time-based functions
Create Calculated Tables
Explore Time-Based Data

Visualizations

Create and Customize Simple Visualizations

Building compelling data visualizations

Identify metrics and pair them with appropriate data visuals
Using slicers
Creating Map Visualizations
Creating Tables and Matrixes
Creating Waterfall and Funnel Charts
Using Gauges and Single Number Cards
Charting Options including Formatting with Colors, Shapes, Text Boxes, Images, etc.

Designing User-friendly reports

Customize themes
Create versatile layouts for your reports
Design principles to reduce noise and highlight data stories

Creating interactive reports for data exploration

Filtering & drilling for insights
Difference between filters & slicers
Filter pane for reporting needs

Power Query

Overview

Introduction
Loading & Refresh
Combine data from multiple data sources

Data Transformation

Editing Queries Created with Power Query
Editing Column Headers in Power Query
Splitting Column Data with Power Query
Sorting Data
Multi-Level Sorting
Filtering Data
Aggregate data from a column
Insert a custom column into a table
Merge columns
Remove columns
Remove rows with errors
Promote a row to column headers
Transforming Text Values
Replacing Data
Using the Fill command
Pivot and Unpivot Column
Transpose Query Data
Pivot Column Command in Action
Unpivot Columns Command
Grouping Data
Create a Duplicate Query
Group and Summarize Data
Advanced Data Grouping
Working with multiple sources in Power Query
Multiple Excel Tables
Expand a column containing an associated table
Understanding Table Relationships
Merging Queries

Loading Power Query Data to Destinations

Familiarity with the Load & Refresh Settings
Loading it to Workbook
Loading it to Data Model

DAX Functions

Overview

What is DAX?
Data Types
Table-Valued Functions
Building a Calendar Table
Date and Time Functions
Filter Functions
Information Functions
Logical Functions
Mathematical and Trigonometric Functions
Statistical Functions
Text Functions
Time Intelligence Functions
Creating Advanced DAX Measures With Advanced DAX Functions

Creating Advanced Dax Measures With Advanced Dax Functions

Calculate()
All()
Filter()
IF()
Switch()
SumX()

Evaluation Context

Filter Context
Row Context
Using RELATED in a Row Context
Filters and Relationships
USERELATIONSHIP

Hierarchies in DAX Querying with DAX Relationships

One-to-Many Relationships
Many-to-Many Relationships

Advanced DAX Functions

ADVANCED DAX FUNCTIONS ADVANCED CONTEXT CONCEPTS

Understanding and Debugging Context Transition
Expanded Tables and Filter Propagation
Using VAR for Performance and Clarity
Using TREATAS() to Apply Filters Between Unrelated Tables
Virtual Relationships using DAX
Shadow Filters and Filter Overriding Techniques

ADVANCED CALCULATION PATTERNS

Running Totals with Custom Filter Logic
Rolling Averages (e.g., 7-day, 12-month)
Year-over-Year (YoY), Quarter-over-Quarter (QoQ), MoM with Non-Standard Calendars
Top N Reporting with Others Grouping
Parent-Child Hierarchy Navigation using PATH, PATHITEM, PATHLENGTH
Budget vs. Actual Comparison Patterns
Custom Grouping (Bucketing) in DAX

ADVANCED TIME INTELLIGENCE

Semi-Additive Measures (e.g., Closing Balance, Opening Balance)
Workdays and Custom Holiday Calendars
Dynamic Period Selection (MTD, QTD, YTD) based on Slicers
Cumulative Totals Across Multiple Tables or Years

ADVANCED FILTER + CALCULATE PATTERNS

Multiple Filters in a Single CALCULATE
Using NOT, EXCEPT, INTERSECT inside CALCULATE
Combine CALCULATE with FILTER(), VALUES(), ALLSELECTED(), KEEPFILTERS()

RELATIONSHIP MODELING TECHNIQUES

Virtual Relationships using DAX (TREATAS, LOOKUPVALUE)
USERELATIONSHIP vs. CROSSFILTER
Handling Bi-Directional Relationships with Care
Many-to-Many Solutions using DAX with Bridge Tables

PERFORMANCE OPTIMIZATION

Understanding and Using DAX Studio and VertiPaq Analyzer
Optimizing Calculated Columns vs. Measures
Query Plans and Storage Engine vs. Formula Engine
Cardinality and its impact on performance
Avoiding common DAX performance anti-patterns (e.g., misuse of SUMX inside FILTER)

DEBUGGING AND TESTING

Using DEFINE MEASURE and EVALUATE in DAX Studio
Evaluating DAX logic step-by-step with VAR and RETURN
Tools: Performance Analyzer in Power BI

SPECIAL FUNCTIONS AND USE CASES

GENERATE(), GENERATEALL(), ADDCOLUMNS()
SUMMARIZE(), SUMMARIZECOLUMNS(), GROUPBY()
ISINSCOPE(), SELECTEDVALUE() vs VALUES()
RANKX(), TOPN(), PERCENTILEX.INC
CONTAINS(), LOOKUPVALUE(), RELATEDTABLE()

SQL Querying

Introduction to MS-SQL

Creating a Database
Understanding Tables and Creating Tables
Inserting, Updating and Deleting Data
Querying Data
Filtering Data
Grouping Data
Ordering Data
Column Aliases
Table Aliases

DDL INSIGHTS

CREATE TABLE
Dropping Objects
CREATE INDEX
TEMPORARY OBJECTS
Object Naming and Dependencies

SELECT STATEMENTS

Simple SELECTs
Calculated and Derived Fields
SELECT TOP / BOTTOM Records
Derived Tables
Joins
Predicates
Subqueries
Aggregate Functions
GROUP BY and HAVING
UNION
ORDER BY

SQL Programming

DDL INSIGHTS

CREATE TABLE
Dropping Objects
TEMPORARY OBJECTS
Object Naming and Dependencies

INTRODUCTION TO SQL PROGRAMMING (T-SQL)

What is T-SQL?
Differences between SQL Querying and SQL Programming
Benefits of procedural SQL
Use cases in reporting, automation, and ETL

VARIABLES AND CONTROL STRUCTURES

Declaring and Using Variables (DECLARE, SET)
Conditional Logic: IF…ELSE
Loops: WHILE, BREAK, CONTINUE
Error Handling: TRY…CATCH, THROW
GOTO statement (rare but useful for certain scenarios)

USER-DEFINED FUNCTIONS (UDFs)

Scalar-Valued Functions
Table-Valued Functions (Inline and Multi-statement)
Best practices for performance
Use in SELECT, WHERE, JOIN clauses

STORED PROCEDURES

Creating and Executing Stored Procedures
Input Parameters, Output Parameters
Reusability and Modularity
Nested Stored Procedures
Use in ETL and Reporting pipelines

TEMPORARY AND TABLE VARIABLES

Temporary Tables (#Temp, ##GlobalTemp)
Table Variables (DECLARE @TableVar TABLE)
Differences, Use Cases, and Scope
CTEs (Common Table Expressions)

CURSORS

Introduction to Cursors
Declaring and Using Cursors
Static vs Dynamic Cursors
Use Cases and Performance Considerations

DYNAMIC SQL

Constructing SQL Statements on the Fly
Executing with EXEC() and sp_executesql

TRANSACTIONS AND ERROR HANDLING

Introduction to Transactions
BEGIN TRAN, COMMIT, ROLLBACK
Nesting Transactions
Isolation Levels
Locking and Blocking

TRIGGERS

AFTER INSERT, AFTER UPDATE, AFTER DELETE
INSTEAD OF Triggers
Auditing Changes
Performance considerations

TESTING AND DEBUGGING

PRINT Statements
RAISERROR for debugging
SQL Server Profiler / Extended Events (if applicable)
Debugging in SQL Server Management Studio (SSMS)

Power BI Administration

Objective: Get hands-on experience with advanced administration settings, permissions, Data refresh times etc.

Overview

Publishing Power BI Reports
Creating & Managing Workspaces and Its Access
Creating & Managing Dashboard and Its Access
Installing & Configuring Data gateway
Scheduling and configuring data refresh
Managing & Reusing Datasets
Scheduling Report Alerts
Setting up Row Level Permissions
Managing Users & Audit Log
Custom Branding Power BI For your Organization
Adding Custom Visuals for your Organization

Power Apps

INTRODUCTION TO POWER APPS

Overview of Power Apps

What is Power Apps?
Benefits and use cases
Types of Power Apps: Canvas, Model-Driven, and Portal Apps

Getting Started

Setting up your Power Apps environment
Navigating the Power Apps interface

DATA INTEGRATION AND MANAGEMENT

Connecting to Data Sources

Introduction to connectors
Connecting to common data sources (SharePoint, Excel, SQL Server, etc.)

Managing Data

Understanding data tables and collections

BUILDING YOUR FIRST CANVAS APP

Basics of Canvas Apps

Understanding Canvas Apps
Creating a simple Canvas App
Adding screens and navigation
Creating simple app to view details from data source.
Using simple forms to display and edit data

ADVANCED CANVAS APP FEATURES

User Experience Design

Designing responsive layouts
Using themes and templates
Best practices for user interface design

Working with Controls

Using different types of controls like
Button
Text input
Drop down
Combo Box
Date picker
List box
Radio
Text label
Vertical gallery
Horizontal gallery
Flexible height gallery
Blank Vertical gallery
Blank Horizontal gallery
Blank Flexible height gallery
Data table
Horizontal container
Vertical container
Container
Image
Icons
Shapes
Working with control properties

Working with Variables

Global variables
Context variable
Collections

Using formulas for dynamic form management

Functions to be used in forms
SubmitForm
EditForm
Clear
ClearCollect
Collect
Filter
If
Navigate
NewForm
Notify
Patch
Refresh
Search
Set
Text
ThisItem
Value

Advanced Controls and Features

Working with media (images, videos)
Implementing charts and graphs
Using Power Automate for workflows

Understanding the Common Data Service (Dataverse)

Creating entities and relationships
Building a simple Model-Driven App

SECURITY AND ADMINISTRATION

Security in Power Apps

Understanding security roles and permissions

Implementing data security

App Management

Managing app versions
Publishing and sharing apps

BEST PRACTICES AND ADVANCED TOPICS

Performance Optimization
Tips for improving app performance
Debugging and troubleshooting techniques
Real-World Use Cases

REALTIME PROJECTS

Project 1: Employee Leave Request App
Project 2: Inventory Management App
Project 3: Customer Feedback App
Project 4: Project Management Dashboard
Project 5: Sales Order Processing App

Power Automate

INTRODUCTION TO POWER AUTOMATE

Overview of Power Automate

What is Power Automate?
Benefits and use cases
Types of flows: Cloud Flows, Desktop Flows, and Business Process Flows

Getting Started

Setting up your Power Automate environment
Navigating the Power Automate interface

CREATING YOUR FIRST FLOW

Basics of Flow Creation

Understanding triggers and actions
Creating a simple flow
Running and testing flows

Flow Templates

Using predefined templates
Customizing template flows
Best practices for using templates

WORKING WITH CONNECTORS

Introduction to Connectors

Understanding connectors and their roles
Connecting to common data sources (SharePoint, OneDrive, Outlook, etc.)

Advanced Data Integration

Using premium connectors
Connecting to SQL Server, Azure, and other advanced data sources

ADVANCED FLOW FEATURES

Conditions and Loops

Implementing conditional logic
Using loops for repetitive tasks

Approvals and Notifications

Creating approval workflows
Sending email and mobile notifications
Error Handling and Troubleshooting

Managing errors in flows

Debugging and troubleshooting techniques

DESKTOP FLOWS (RPA)

Introduction to Desktop Flows

Understanding Robotic Process Automation (RPA)
Setting up Power Automate Desktop

Building Desktop Flows

Recording desktop actions
Automating desktop applications

Advanced Desktop Flow Features

Using conditions and loops in desktop flows
Integrating with cloud flows

BUSINESS PROCESS FLOWS

Introduction to Business Process Flows

Understanding business process automation
Creating a simple business process flow

Customizing Business Process Flows

Defining stages and steps
Implementing business rules and logic

Advanced Business Process Flow Features

Using custom entities and fields
Integrating with Power Apps

PROJECTS

Power Automate Projects

Project 1: Automated Invoice Approval Workflow
Project 2: Employee Onboarding Automation
Project 3: Social Media Post Scheduler
Project 4: Customer Support Ticketing System
Project 5: Monthly Sales Report Automation

Desktop Flows (RPA)

Project 1: Automated Data Entry from Emails
Project 2: Invoice Processing and Archiving
Project 3: Automated Report Generation
Project 4: Customer Account Reconciliation
Project 5: Automated Data Migration

Python for Automation

SSIS

SSAS

Azure Blob Storage

Introduction to Azure Blob Storage

Overview of Azure Storage Services
What is Azure Blob Storage?
Use Cases of Blob Storage (Data Archiving, Backup, Big Data Analytics, etc.)
Types of Azure Storage Accounts

Understanding Azure Blob Storage Architecture

Containers, Blobs, and Storage Accounts

Types of Blobs:

Block Blobs
Append Blobs
Page Blobs

Storage Tiers:

Hot
Cool
Archive

Data Replication Strategies:

LRS (Locally Redundant Storage)
ZRS (Zone-Redundant Storage)
GRS (Geo-Redundant Storage)
RA-GRS (Read Access Geo-Redundant Storage)

Setting Up and Managing Azure Blob Storage

Creating an Azure Storage Account
Creating and Configuring Containers
Uploading, Downloading, and Managing Blobs
Managing Access Control:
Shared Access Signatures (SAS)
Azure Active Directory Authentication
Role-Based Access Control (RBAC)
Storage Account Keys

Working with Azure Blob Storage using Tools

Using Azure Portal for Blob Storage Management
Working with Azure Storage Explorer
Managing Blob Storage with Azure CLI
Automating Blob Operations with PowerShell

Azure Blob Storage Integration with BI & Automation

Integrating Blob Storage with Power BI:

Connecting Power BI to Azure Blob Storage
Using Dataflows to Process Blob Data

Using Blob Storage with Azure Data Factory:

Copying Data from Blob Storage to Azure SQL Database
Data Transformation using Mapping Data Flows
Automating Data Processing with ADF Pipelines
Azure Logic Apps for Automating Blob Storage Workflows
Azure Synapse Analytics Integration with Blob Storage for Big Data Processing

Advanced Azure Blob Storage Features

Soft Delete, Versioning, and Snapshot Management
Data Lifecycle Management and Cost Optimization
Azure Storage Encryption and Security
Cross-Origin Resource Sharing (CORS)
Event Grid and Blob Storage Event Handling
Using Azure Data Lake Gen2 with Blob Storage

Hands-on Projects & Real-World Scenarios

Project 1: Setting up a Data Lake using Azure Blob Storage for Power BI Analytics
Project 2: Automating Data Uploads from Power Automate to Azure Blob Storage
Project 3: Creating an ETL Pipeline using Azure Data Factory with Blob Storage
Project 4: Implementing Data Retention Policies using Azure Blob Storage Lifecycle Management

Azure Data Lake

Introduction to Azure Data Lake

What is Azure Data Lake?
Difference Between Azure Data Lake and Azure Blob Storage
Data Lake Gen1 vs Gen2
Key Benefits and Use Cases

Azure Data Lake Architecture

Understanding the Hierarchical Namespace
Components of Data Lake:
Storage Accounts
Containers & Folders
Files & Metadata
Security Model & Data Access
Data Lake File Format Best Practices (Parquet, CSV, JSON, Avro)

Setting Up Azure Data Lake Storage (ADLS Gen2)

Creating an Azure Storage Account
Configuring Azure Data Lake Gen2
Managing Storage Containers & Files
Authentication & Authorization:
Azure Active Directory (Azure AD)
Role-Based Access Control (RBAC)
Access Control Lists (ACLs)

Working with Azure Data Lake Using Tools

Using Azure Portal for Data Lake Management
Azure Storage Explorer for File Operations
Azure CLI & PowerShell for Automation
Accessing Data with Python & Pandas
Azure Synapse Studio for Data Lake Exploration

Data Ingestion into Azure Data Lake

Ingesting Data with Azure Data Factory (ADF)
Copy Data from SQL Server, Blob Storage, REST API
Scheduling & Monitoring Pipelines
Power Automate Integration
Streaming Data into Data Lake with Azure Event Hub

Data Processing & Transformation in Data Lake

Using Azure Synapse Analytics with Data Lake
Querying Data with Serverless SQL Pools
Using Spark Pools for Big Data Processing
Transforming Data with Dataflows in Power BI
ETL Pipelines using Azure Data Factory Mapping Data Flows
Integrating with Azure Databricks for Advanced Processing

Securing & Monitoring Azure Data Lake

Encryption at Rest & In-Transit
Azure Defender for Storage
Data Masking & Sensitive Data Protection
Auditing & Logging with Azure Monitor

Real-World Projects & Use Cases

Project 1: Creating a Data Lake for Power BI Reporting
Project 2: Building an ETL Pipeline with Azure Data Factory & Data Lake
Project 3: Integrating Data Lake with Azure Synapse for Advanced Analytics
Project 4: Automating File Processing with Power Automate & Data Lake

Azure Data Factory

Introduction to Azure Data Factory (ADF)

What is Azure Data Factory?
Key Features and Benefits
Understanding Data Integration and ETL in Azure
Comparing ADF with SSIS and Other ETL Tools

Understanding ADF Architecture

Components of Azure Data Factory:

Pipelines
Activities
Datasets
Linked Services
Triggers
Integration Runtimes (Self-hosted vs Azure-Hosted)
Data Movement vs Data Transformation in ADF

Setting Up Azure Data Factory

Creating an Azure Data Factory Instance
Navigating the Azure Data Factory UI
Understanding the Author, Monitor, and Manage Sections
Connecting to On-Premises and Cloud Data Sources

Data Ingestion with Azure Data Factory

Copy Data Tool for Quick Ingestion
Connecting to Data Sources:
Azure Blob Storage / Data Lake
SQL Server, Azure SQL Database
REST APIs & Web Services
On-Premises Databases with Self-Hosted Integration Runtime
Incremental Data Load with Watermarking

Data Transformation in ADF

Using Mapping Data Flows (No-Code Transformations)
Data Transformation Activities
Data Flow Debugging and Monitoring
Using Wrangling Data Flows (Power Query in ADF)
Calling Azure Databricks for Advanced Transformations
Executing Stored Procedures for Data Processing

Orchestrating Data Pipelines

Pipeline Execution and Debugging
Control Flow Activities:
ForEach and Until Loops
If Condition & Switch Activity
Wait, Lookup, and Execute Pipeline Activities
Error Handling and Logging Strategies

Scheduling & Monitoring Pipelines

Triggers in ADF:
Tumbling Window Trigger
Schedule-Based Trigger
Event-Based Trigger
Monitoring Pipelines with Azure Monitor
Logging and Error Handling in ADF
Managing Pipeline Versions and Change Tracking

Integrating Azure Data Factory with Other Services

Power BI: Ingesting and Processing Data for Reporting
Azure Logic Apps: Automating Pipeline Execution
Azure Data Lake & Blob Storage: Managing Large Data Loads
Azure Synapse Analytics: ELT Pipeline for Big Data Processing

Real-World Projects & Hands-on Labs

Project 1: ETL Pipeline – Ingesting Data from SQL Server to Azure Data Lake
Project 2: Transforming and Cleansing Data with Mapping Data Flows
Project 3: Automating Data Pipelines with ADF Triggers
Project 4: Building a Hybrid Data Pipeline using Self-Hosted Integration Runtime

Azure Data Bricks

Introduction to Azure Databricks

What is Azure Databricks?
Key Features and Benefits
Azure Databricks vs Azure Synapse vs HDInsight
Use Cases: Big Data Processing, AI/ML, Data Science, ETL Pipelines

Azure Databricks Architecture & Components

Understanding Databricks Workspace
Clusters: Types, Auto-scaling, and Configuration
Notebooks: Using Python, Scala, SQL & R
Jobs: Automating Workflows
Libraries: Managing Dependencies (PySpark, MLflow, Delta Lake)

Setting Up Azure Databricks

Creating an Azure Databricks Workspace
Navigating the Databricks UI
Configuring and Launching Clusters
Managing Users & Permissions (RBAC & ACLs)

Working with Databricks Notebooks

Writing Code in Databricks (Python, SQL, Scala)
Using Magic Commands (%sql, %python, %scala)
Importing and Exporting Data
Collaborative Development in Notebooks
Visualizing Data with Built-in Charts

Data Engineering with Databricks & Spark

Introduction to Apache Spark in Databricks
DataFrames & Spark SQL for Data Processing
Connecting to Azure Blob Storage & Data Lake
Reading and Writing Parquet, JSON, CSV Files
Performance Optimization with Partitions & Caching

ETL Pipelines with Azure Databricks

Building ETL Pipelines Using PySpark
Delta Lake for ACID Transactions & Schema Evolution
Integrating Databricks with Azure Data Factory
Automating Pipelines Using Databricks Jobs & Triggers
Error Handling & Logging in ETL Pipelines

Real-World Projects & Hands-on Labs

Project 1: Building a Data Pipeline Using Azure Databricks & Delta Lake
Project 2: ETL Pipeline with Azure Data Factory & Databricks
Project 3: Real-time Data Processing with Azure Event Hub & Databricks
Project 4: Deploying an ML Model Using Databricks & MLflow

Azure Logic App

Introduction to Azure Logic Apps

What is Azure Logic Apps?
Key Features & Benefits
Logic Apps vs Power Automate vs Azure Functions
Common Use Cases:
Process Automation
Data Integration
Workflow Orchestration
Event-Driven Automation

Azure Logic Apps Architecture & Components

Triggers: Event-based, Recurrence-based, Manual Triggers
Actions: Built-in & Custom Actions
Connectors: Standard vs Premium
Workflows: Stateful vs Stateless
Run History & Monitoring

Setting Up Azure Logic Apps

Creating a Logic App in Azure Portal
Navigating the Logic Apps Designer
Understanding the Designer vs Code View
Creating and Managing Resource Groups
Deploying Logic Apps using Azure CLI & ARM Templates

Working with Triggers & Actions

Triggers:
HTTP Request & Response
Recurrence & Schedule-Based
Event Grid & Service Bus
File System, OneDrive, SharePoint, Blob Storage Triggers
Actions:
HTTP Calls & API Requests
Database Operations (SQL, CosmosDB)
Message Queues (Azure Service Bus, Event Hub)
Looping & Conditional Statements
Error Handling & Retry Policies

Connecting Logic Apps with Microsoft Services

Power Automate & Power BI: Automating BI Reports
Power Apps Integration: Triggering Workflows from Apps
Azure SQL Database & SharePoint: Syncing Data
Microsoft 365 & Outlook: Automating Email Workflows
Microsoft Teams & Notifications

Working with APIs & External Services

Calling REST APIs from Logic Apps
Using API Management (APIM) with Logic Apps
Authentication & Security in API Calls
OAuth & API Keys for External Services

Advanced Logic App Workflows

Stateful vs Stateless Workflows
Parallel Processing & Batch Execution
Long-Running Workflows & Durable Functions
Using Expressions & Variables in Logic Apps
Using For-Each, Switch, and Do-Until Loops

Real-World Projects & Hands-on Labs

Project 1: Automating Email Notifications for Business Users
Project 2: Integrating Azure Blob Storage with SQL Database
Project 3: Syncing Power BI Reports with SharePoint Data
Project 4: Connecting Logic Apps with Service Bus for Messaging

Azure Synapse Anaytics

Introduction to Azure Synapse Analytics

What is Azure Synapse Analytics?
Synapse vs Azure SQL Database vs Azure Databricks
Key Features & Benefits
Understanding Data Warehousing vs Big Data Analytics
Common Use Cases:
Data Warehousing
Real-Time Data Analytics
ETL/ELT Pipelines
BI & Reporting

Understanding Synapse Architecture

Synapse SQL Pools:
Dedicated SQL Pools vs Serverless SQL Pools
Synapse Pipelines:
ETL & Data Integration
Synapse Studio:
Data Exploration & Querying
Data Integration with Linked Services
Integration with Power BI, Azure Data Factory, and Databricks

Setting Up Azure Synapse Analytics

Creating an Azure Synapse Workspace
Navigating the Synapse Studio UI
Understanding Dedicated vs Serverless SQL Pools
Configuring Linked Services (Azure Blob, Data Lake, SQL Server, etc.)
Role-Based Access Control (RBAC) & Security

Ingesting Data into Azure Synapse

Using Azure Data Factory to Load Data into Synapse
Copying Data from On-Premises & Cloud Sources
Connecting to Azure Blob Storage & Data Lake
Using T-SQL and PolyBase for Data Ingestion
Delta Lake Integration for Big Data Processing

Querying & Managing Data in Synapse SQL Pools

Working with Serverless SQL Pools
Writing Queries with T-SQL & Spark SQL
Performance Optimization with Partitioning & Indexing
Columnstore Indexes for Faster Queries
Best Practices for Writing SQL Queries in Synapse

Data Transformation & ETL in Synapse

Using Data Flows in Synapse Pipelines
Transforming Data with Spark Notebooks
Executing Stored Procedures for ETL Processing
Automating Data Pipelines with Triggers & Scheduling
Integrating Synapse Pipelines with Power Automate

Power BI Integration with Azure Synapse

Connecting Power BI to Synapse SQL Pools
Optimizing Power BI Reports with Synapse Data
Using Synapse as a Data Source for Power BI Dataflows
DirectQuery vs Import Mode Considerations

Real-World Projects & Hands-on Labs

Project 1: Building a Data Warehouse in Azure Synapse
Project 2: ETL Pipeline with Azure Synapse & Data Factory
Project 4: Power BI Reporting using Synapse SQL Pools

Azure SQL Database

Introduction to Azure SQL Database

What is Azure SQL Database?
Azure SQL vs SQL Server vs Synapse Analytics vs Cosmos DB
Key Features & Benefits
Common Use Cases:
Cloud-based Relational Database Management
High Availability & Scalability
BI & Analytics Integration

Azure SQL Database Deployment Options

Single Database vs Elastic Pools vs Managed Instance
Understanding DTUs vs vCores
Choosing the Right Service Tier (Basic, General Purpose, Business Critical, Hyperscale)
Serverless vs Provisioned Compute Model

Setting Up an Azure SQL Database

Creating an Azure SQL Database using Azure Portal, PowerShell, and CLI
Configuring Firewall & Network Security
Connecting to SQL Database using SSMS, Azure Data Studio, and Power BI
Role-Based Access Control (RBAC) & Authentication

Working with Azure SQL Database

Writing SQL Queries using T-SQL
Creating and Managing Tables, Views, and Stored Procedures
Understanding Indexes, Constraints & Triggers
Transactions & Locking Mechanisms
Querying with JSON & XML Data

Data Ingestion & ETL Pipelines

Using Azure Data Factory to Load Data into SQL Database
Bulk Insert, BCP & COPY Commands for Large Data Loads
Using Power Automate to Automate Data Entry
Connecting Azure SQL Database with Power BI for Analytics

Performance Optimization & Query Tuning

Indexing Strategies (Clustered vs Non-Clustered, Columnstore Indexes)
Query Store for Performance Monitoring
Execution Plans & Query Tuning Techniques
Partitioning Tables for Large Datasets
Caching & Optimizing Reads with Materialized Views

Integrating Azure SQL Database with Other Azure Services

Power BI: DirectQuery vs Import Mode for SQL Database
Azure Data Factory: ETL Pipeline Development
Azure Logic Apps & Power Automate: Automating Workflows
Azure Synapse Analytics: Exporting Data for Analytics
Azure Functions: Triggering Events from SQL Database

Real-World Projects & Hands-on Labs

Project 1: Migrating an On-Premises SQL Database to Azure
Project 2: Automating Data Entry Using Power Automate & Azure SQL
Project 3: Building an ETL Pipeline Using Azure Data Factory
Project 4: Creating a Power BI Dashboard Using Azure SQL Database

Azure Stream Analytics

Introduction to Azure Stream Analytics (ASA)

What is Azure Stream Analytics?
How Stream Analytics fits into the Azure Data Ecosystem
Key Features & Benefits
Batch vs Stream Processing: Understanding the Differences
Common Use Cases:
Real-time Data Processing
IoT Data Streams
Event-Driven Analytics
Fraud Detection & Monitoring

Understanding Azure Stream Analytics Architecture

Input Sources:
Azure Event Hubs
Azure IoT Hub
Azure Blob Storage
Query Engine & Processing:
T-SQL-Based Querying
Temporal Windowing Functions
Output Destinations:
Power BI for Real-time Dashboards
Azure SQL Database
Azure Blob Storage
Azure Data Lake
Azure Synapse Analytics

Setting Up an Azure Stream Analytics Job

Creating an ASA Job from the Azure Portal
Configuring Input Sources (Event Hubs, IoT Hub, Blob Storage)
Writing a Basic Stream Query
Defining Output Destinations
Running & Debugging a Stream Analytics Job

Writing Stream Analytics Queries

Introduction to Stream Analytics Query Language (SAQL)
Filtering & Transformation Queries
Aggregating Data in Real Time
Using Windowing Functions:
Tumbling Windows
Sliding Windows
Hopping Windows
Session Windows
Joining Multiple Streams
Detecting Anomalies & Patterns

Connecting Stream Analytics to Power BI

Streaming Data to Power BI Dashboards
Building Real-Time Reports in Power BI
Optimizing Power BI for Streaming Data
Use Cases: IoT Data Monitoring, Live Business Metrics

Advanced Stream Processing Techniques

User-Defined Functions (UDFs) in JavaScript & C#
Integrating Azure Machine Learning for Predictive Analytics
Using Reference Data for Enriching Streams
Error Handling & Retries in Stream Analytics
Event Processing with Azure Functions

Real-World Projects & Hands-on Labs

Project 1: Streaming IoT Sensor Data to Azure SQL Database
Project 2: Real-time Fraud Detection Using ASA & Machine Learning
Project 3: Live Monitoring Dashboard with Azure Stream Analytics & Power BI
Project 4: Processing Social Media Feeds for Sentiment Analysis

Azure Cosmos DB

Azure Analysis Services / Power BI Dataset

Power BI Service

System Requirements

System Requirements:

Power BI Desktop

Free & Downloadable from Microsoft Store App

https://www.microsoft.com/store/productid/9NTXR16HNW1T?ocid=pdpshare
Excel 2016 & above with PowerPivot

Available with Office 365 subscriptions that include desktop versions of Excel for Windows.
MS-SQL Server

https://www.microsoft.com/en-in/sql-server/sql-server-downloads

Kindly install Developer Edition as it is full-featured free edition, licensed for use as a development and test database in a non-production environment.
SQL Server Management Studio (SSMS)

Download and install SSMS from https://learn.microsoft.com/en-us/sql/ssms/download-sql-server-management-studio-ssms?view=sql-server-ver16
Secondary Monitor (optional, but recommended)

Having a secondary monitor will greatly assist in following the pace of the trainer. It allows you to view instructions and your own workspace simultaneously, enhancing your learning experience.

Taught by Microsoft Certified Trainers

All our classes are live,
hands-on and with
real-trainers.

End-to-End Data Engineering & BI Projects

Real-time Projects (Practical Application)

Bridge the gap between legacy systems and modern cloud solutions with hands-on projects in Power BI, SQL, Azure Data Factory, Databricks, Synapse, Power Apps, Power Automate, VBA, and Python. Gain real-world experience in ETL, data modeling, automation, and analytics to transition seamlessly from On-Prem to Cloud BI and build scalable, enterprise-grade reporting solutions.

Power BI

SQL

VBA & Python

ETL

Data Factory & Databricks

Azure Synapse & SQL

Power Apps

Power Automate

Master Projects

Enterprise Sales Performance Dashboard

Develop an interactive Power BI dashboard to analyze regional and product-wise sales performance. Utilize DAX measures for YOY growth, sales variance, and customer segmentation, while Power Query is used to clean and transform raw sales data from multiple sources.

Customer Churn Prediction

Build a churn prediction model by integrating customer interaction data, support tickets, and transactional history. Use Power Query for data transformation and DAX measures to calculate churn probability based on behavioral patterns.

HR Analytics & Workforce Planning

Create an HR dashboard to track employee retention, hiring trends, and performance metrics. Connect multiple data sources to Power BI, apply DAX formulas to calculate attrition rates, and implement role-based security for restricted views.

Supply Chain & Inventory Optimization

Design a Power BI solution to monitor inventory levels, supplier performance, and stock movement across locations. Use Power Query to merge purchase order data with warehouse stock levels and apply DAX for predictive analytics on stock replenishment.

Financial Reporting & Budget vs. Actuals Analysis

Automate financial reporting using DAX calculations for variance analysis, custom KPIs, and trend forecasting. Apply Power Query transformations to consolidate financial statements from different departments into a unified report.

Database Design & Normalization

Design and normalize relational databases using primary keys,foreign keys, and normal forms to optimize storage and scalability.

Customer Data Analysis

Analyze customer data with JOINs, GROUP BY, and HAVING to identify trends, segment groups, and generate insights for marketing.

Sales Data Analysis

Use window functions and CTEs to track sales metrics, revenue, and product performance, generating actionable insights.

Inventory Management Queries

Leverage subqueries, UNION, and CASE to track inventory, product stock, and sales velocity for decision-making.

Employee Performance Tracker

Apply ranking functions (e.g., RANK(), DENSE_RANK()) to evaluate employee performance, track KPIs, and enhance workforce management.

Financial Reporting Queries

Retrieve financial data using aggregate functions and GROUP BY to automate real-time reports for balance sheets and P&L statements.

Automated Excel Report Generation

Automate the creation of complex reports with VBA macros and Python scripts. Generate dynamic pivot tables, charts, and formatted summaries at the click of a button.

Email & Report Automation

Use VBA and Outlook automation to send scheduled reports based on data conditions. Ensure timely delivery of financial, sales, or operational updates.

Web Scraping & Data Collection

Automate data extraction from websites using Python (BeautifulSoup, Selenium) to collect real-time insights for competitive analysis and trend tracking.

Data Cleaning & Transformation

Leverage Python and VBA to clean and standardize raw data, automate missing value handling, and prepare datasets for reporting.

Smart Financial Generator

Use VBA and Python to extract financial data, consolidate income statements, and generate automated balance sheets with dynamic updates.

Sales & Customer Data Pipeline

Extract, clean, and load customer and sales data from multiple sources using Power Query and SQL, ensuring real-time reporting accuracy.

Employee Data Warehouse

Automate ETL workflows to consolidate employee records, payroll, and performance data, creating a structured HR analytics database.

Automated Data Migration & Cleansing

Transform and migrate legacy system data into modern databases, ensuring integrity and accuracy with Python and Power Query automation.

Marketing Data Consolidation

Integrate data from multiple marketing platforms (Google Ads, Facebook, LinkedIn) into a structured dataset, enabling real-time campaign performance tracking.

Inventory & Procurement Data Pipeline

Automate inventory tracking and supplier performance analysis with SQL and Python, setting up alerts for low stock levels and procurement optimization.

Automated Data Pipeline for Reporting

Build an end-to-end ETL pipeline using Azure Data Factory (ADF) to ingest, clean, and transform structured and unstructured data from multiple sources. Ensure automated report updates in Power BI for real-time business insights.

Incremental Data Load for Real-Time Analytics

Implement incremental data loading in ADF to update a cloud data warehouse in real time. Use time-based partitioning to handle high-frequency data changes efficiently while ensuring data consistency.

ETL Workflow Automation

Design a fully automated ETL process in ADF that extracts data from APIs, SQL databases, and flat files, transforms it using Databricks, and loads it into Azure Synapse Analytics for advanced business intelligence reporting.

Automated Data Quality & Governance Framework

Develop a data validation and anomaly detection framework using ADF and Databricks. Implement logging mechanisms and alerts to detect inconsistencies, ensuring high data integrity for analytics.

Data Warehousing & Performance Optimization

Design a star schema-based data warehouse in Azure Synapse to store structured business data efficiently. Implement partitioning, indexing, and materialized views to optimize query performance for large datasets.

Customer 360-Degree View

Integrate multiple structured and unstructured data sources (CRM, web analytics, transaction logs) into Azure Synapse to build a unified customer profile. Use SQL to perform data cleansing, deduplication, and customer segmentation analysis.

Real-time ETL with SQL Pools

Implement an ETL workflow in Synapse SQL pools to process and transform real-time streaming data. Apply SQL transformations and load aggregated insights into Power BI for dashboard visualization.

Predictive Analytics with Big Data

Use SQL queries and Synapse’s machine learning integration to generate predictive insights on customer behavior, seasonal sales trends, and fraud detection. Optimize query execution using columnstore indexes and parallel processing.

Employee Leave Request App

Design a user-friendly app for employees to submit leave requests. Automate approval workflows, track balances, and integrate with SQL databases for real-time leave management.

Sales Order Management App

Build an interactive app for sales teams to create, update, and track orders. Integrate with SQL to retrieve customer data and Power Automate to generate invoices and email confirmations.

Helpdesk System

Develop a mobile-friendly app where customers can log support tickets. Use Power Automate to assign cases, send updates, and generate Power BI reports for service performance tracking.

Expense Reimbursement Tracker

Create an app for employees to submit expense claims with receipt uploads. Automate approval workflows, validate entries against company policies, and generate real-time reimbursement status updates.

Inventory & Asset Management App

Design a system to track inventory and company assets, with barcode scanning capabilities. Connect to a SQL database for real-time stock updates and automate notifications for low inventory levels.

Automated Report Distribution

Schedule and distribute Power BI reports to stakeholders via email or Microsoft Teams. Trigger updates based on data refresh cycles or predefined conditions.

Customer Onboarding Workflow

Streamline new customer onboarding by automating email confirmations, database entry creation in SQL, and document approvals through SharePoint integration.

Invoice Processing & Approval

Extract data from email attachments, validate invoice details against SQL records, and automate approval requests via Teams or Outlook.

Automated Data Entry & Sync

Integrate multiple data sources (Excel, SharePoint, SQL) to automate data entry and keep records synchronized across different systems.

Real-Time Alerts & Notifications

Set up automated alerts for critical business events, such as exceeding sales targets, system downtime, or low inventory, with notifications sent via Teams, email, or mobile push alerts.

Enterprise Data Lake & BI Ecosystem

Develop an end-to-end BI ecosystem using Azure Data Lake, Synapse Analytics, and Power BI. Implement data ingestion, transformation, storage, and visualization for enterprise-wide reporting.

AI-Powered Sales Forecasting & Reporting

Build a forecasting model using Databricks and integrate it with Synapse SQL for real-time analytics. Visualize predictive insights in Power BI dashboards to assist sales teams in strategic decision-making.

End-to-End Cloud-Based Financial Analytics

Design a full-scale automated financial reporting system integrating ADF, Synapse, Power BI, and Logic Apps to provide real-time financial insights, budget tracking, and regulatory compliance reporting.

Design and normalize relational databases using primary keys, foreign keys, and normal forms to optimize storage and scalability.