Professional Certification in Data Engineer & Gen Al

Introduction to Computer Programming (Python)

• Programming Foundation Concepts
  • Basic Terminology
  • Data Types and Variables
  • Operators and Expressions
  • Control Structures
  • Functions and Procedures
  • Recursion
  • Basic Algorithms
• Object-Oriented Concepts
  • Classes and Objects
  • Inheritance
  • Polymorphism
  • Encapsulation
  • Abstraction
  • Method Overriding
  • Constructor and Destructor
• Getting Started with Python
  • History of Python
  • Python Features
  • Python 2 vs Python 3
  • Installing Python
  • Running Python Scripts
  • Python Syntax and Semantics
  • Python Development Environments
• Python Installation and Setup
  • Python 3.X Installation Guide
  • Introduction to Anaconda Python
  • IDEs: Jupyter Notebook, PyCharm, and VS Code
  • Setting Up Virtual Environments
  • Package Management with pip and conda
• Python Built-in Objects
  • Numbers (int, float, complex)
  • Strings
  • Lists
  • Tuples
  • Sets
  • Dictionaries
  • Type Conversion
• Control Flow in Python
  • Conditional Statements (if, elif, else)
  • Looping Statements (for, while)
  • Nested Loops
  • Loop Control Statements (break, continue, pass)
  • Comprehensions (List, Dict, Set)
  • Conditional Expressions
• Functions in Python
  • User Defined Functions
  • Lambda Functions
  • Function Arguments (default, keyword, arbitrary)
  • Return Statement
  • Recursion
  • Built-in Functions
  • Higher-Order Functions
• String Handling in Python
  • String Creation and Operations
  • String Methods
  • String Formatting
  • Regular Expressions
  • String Slicing
  • Escape Characters
  • Unicode Strings
• Data Structures in Python
  • List Operations and Methods
  • Dictionary Operations and Methods
  • Set Operations and Methods
  • Tuple Operations and Methods
  • Working with Stacks and Queues
  • Comprehensions
  • Collections Module
• File Handling in Python
  • Reading and Writing Files
  • File Methods
  • File Modes
  • Handling Binary Files
  • Working with CSV Files
  • Context Managers
  • File Path Operations
• Python Libraries and Modules
  • Standard Library Overview
  • Commonly Used Libraries (os, sys, math, datetime)
  • Importing Modules
  • Creating User Defined Modules
  • User Defined Packages
  • Working with init.py
  • Packaging and Distribution (whl Packaging)
• Advanced Python Concepts
  • Object Oriented Python
  • Exception Handling
  • Debugging Techniques
  • Iterators and Generators
  • Decorators
  • Context Managers
  • Metaprogramming
• Database Interface with Python
  • Introduction to Databases
  • SQLite in Python
  • CRUD Operations
  • Connecting to Sqlite
  • Database Connection Pooling
  • Handling Transactions
• Working with Data Formats
  • Introduction to JSON Data
  • Parsing JSON
  • Creating JSON
  • Working with XML
  • Using CSV Module
  • pickle Module for Serialization
  • YAML Format
• Web Programming with Python
  • Requests Module
  • Web Scraping with BeautifulSoup
  • Using Scrapy for Advanced Scraping
  • Working with APIs
• Numerical Computation with Numpy
  • Introduction to Numpy
  • Numpy Arrays
  • Array Indexing and Slicing
  • Array Operations
  • Broadcasting
  • Numpy Functions
  • Working with Matrices
• Data Manipulation with Pandas
  • Introduction to Pandas
  • Series and DataFrame
  • Reading and Writing Data
  • Data Cleaning
  • Data Transformation
  • Merging and Joining
  • Data Aggregation

Introduction to RDBMS Database (SQL)

• DATA DOMAIN – WAREHOUSING & SQL
  • SQL Basics for Analysis
  • Query Fundamentals
  • Multiple Tables
  • Advanced Joins
  • Type Casting & Math
  • Date Time & String
  • Window Functions
  • CTE & Pivot
  • Data Exploration with SQL & Python
  • DB Mgmt & Schema Design
• SQL+ETL+DATA WAREHOUSING – DBT
  • Dimensional modeling
  • dbt project structure (staging/core/marts)
  • SQL+Jinja, macros, variables
  • materializations & configs
  • Incremental models & MERGE
  • Snapshots
  • Generic/singular tests & data contracts
  • docs site & exposures; slim CI/state
  • adapters (dbt-databricks / dbt-sqlserver)
• SPARK – PYSPARK – FOUNDATIONS
  • Spark architecture
  • DataFrames API
  • UDF vs pandas UDF
  • Spark SQL/catalogs
  • Delta-style concepts
  • Performance: partitioning, caching, broadcast joins, AQE, skew fixes
  • Testing PySpark jobs
• ORCHESTRATION – APACHE AIRFLOW
  • DAGs, scheduling, retries & SLAs
  • Operators, Sensors, XCom
  • TaskFlow API
  • params & templating
  • connections & secrets
  • testing DAGs (DagBag/pytest), backfills;
  • observability & alerts
  • best practices
• AZURE DATA ENGINEERING PREP
  • ADLS Gen2 (HN, lifecycle, security)
  • ADF & Synapse Pipelines
  • Azure Databricks (jobs, repos)
  • Synapse (serverless vs dedicated)
  • Serving with serverless SQL
  • Security (RBAC, ACLs, private endpoints, Key Vault)
  • Monitoring & cost
  • Purview (catalog, lineage)
  • Exam strategy
• GENERATIVE AI
  • Foundations of LLMs & Prompt Engineering
    • Fundamentals of LLMs
    • Understand how LLMs work, including tokenization and embeddings.
    • Explore how models like GPT interpret and generate text.
    • Project: Visualize and compare embeddings.
    • Prompt Engineering in Practice
    • Learn how to structure and format prompts effectively.
    • Understand chat roles: system, user, assistant.
    • Practice designing multi-turn prompts and chaining logic.
    • Project: Create a multi-turn chatbot.
  • Prompt Engineering with OpenAI API
    • Apply prompting techniques for summarizing, classifying, transforming text.
    • Use OpenAI API to test and iterate prompts programmatically.
    • Project: Implement and compare zero-/one-/few-shot prompts for text Summarization, sentiment detection, and email drafting.
  • Building Systems using Generative AI
    • Apply prompt engineering to solve real-world text tasks.
    • Split complex tasks into a pipeline of subtasks using multi-stage prompts.
    • Evaluate LLM inputs and outputs for safety, accuracy, and relevance.
    • Project: Build a customer-support agent that can flag, retrieve and answer user queries.
  • LangChain & Agent Systems
    • Application Development using LangChain
    • Understand core LangChain framework.
    • Learn how to use your own data with RAG (Retrieval-Augmented Generation).
    • Build chatbots powered by structured chaining logic.
    • Project: Create a document Q&A assistant using LangChain and your own PDFs.
  • Multi-Agent Systems with AutoGen
    • Learn to scale LLM performance through Agentic AI systems using AutoGen.
    • Compare agent patterns such as coordinator-worker, role-based agents, and tool-invoking agents.
    • Explore real-world task automation like document processing or customer service escalation.
    • Optional: contrast AutoGen with CrewAI for understanding trade-offs.
    • Project: Build a 3-agent AutoGen system to automate customer support (intake, triage, escalation).
  • Agentic Design with LangGraph
    • Learn to design structured agent workflows with LangGraph.
    • Combine LangGraph’s visual DAG-based control with AutoGen’s conversational memory.
    • Project: Implement a LangGraph / AutoGen hybrid agent that dynamically routes tasks and retries failed steps.
  • Reasoning with LLMs
    • Learn how reasoning models work and when to use them.
    • Use chain-of-thought and self-reflection prompting techniques.
    • Apply reasoning to code and vision-based tasks.
    • Project: Use reasoning prompts to debug code and answer image-based questions step-by-step.
  • Reasoning Models & Advanced Agents
    • Advanced Use Cases with AutoGen
    • Design collaborative multi-agent systems using AutoGen’s planner-executor framework.
    • Project: Build an AutoGen-based resume screening system.
  • Build with Anthropic – Computer Use
    • Understand Anthropic family of models.
    • Implement low-latency and cost-saving techniques like prompt caching.
    • Project: Build an AI assistant that opens a file, summarizes content, and suggests edits.
  • Reasoning + Agent Project Sprint
    • Project: Build a Virtual Analyst using LangGraph and AutoGen.
  • Memory, Context Protocols & AI Safety
    • Long-Term Memory with LangGraph + AutoGen
    • Model Context Protocol (MCP)
    • Project: Deploy an MCP server and connect mini-apps.
  • Guardrails for AI Systems
    • Recognize common failure modes in LLM systems.
    • Use guardrails to validate and filter model inputs and outputs.
    • Project: Add guardrails to a Generative AI app.
  • AI Project Best Practices
    • End-to-end AI application lifecycle.
    • Prompting vs fine-tuning trade-offs.
    • Evaluation, deployment, and monitoring.