search

HydPy / HydPy-meetups

CFP manager for HydPy meetups
MIT License
27 stars 21 forks source link

Airflow: essentials of workflow orchestrator for ETL pipelines #71

Open sanghisha145 opened 5 months ago

sanghisha145 commented 5 months ago

Title of the talk/workshop Airflow: essentials of workflow orchestrator for ETL pipelines

Abstract of the talk/workshop Apache Airflow has emerged as a powerful tool for orchestrating and automating data workflows. This talk will introduce participants to the core concepts of Apache Airflow, including DAGs, operators, schedulers, and its UI. We will explore practical examples of setting up workflows, monitoring tasks, and leveraging Airflow for efficient data pipeline management. Whether you're new to Airflow or looking to deepen your understanding, this session aims to equip you with the knowledge to streamline your data engineering processes.

Category of the talk/workshop Data Engineering

Duration (including Q&A) 30min

Level of Audience Beginner

Speaker Bio

  1. Speaker Bio (Brief) -> Divya Sanghi is a senior software engineer who is interested in Big Data and processing, cloud computing, containerization, distributed systems, design system.
  2. Company/College -> Walmart Labs
  3. Email -> divyasanghi145@gmail.com
  4. Years of Exp ->3

Prerequisites(if any)

  1. standalone airflow installed
  2. python version 3.8+
bpkapkar commented 1 month ago

@sanghisha145 We are planning a meetup on October 19th and would love to know your availability for an in-person gathering in Hyderabad. This will help us organize the event accordingly.