Build an ETL pipeline in Apache Airflow

[Maniktherana]

Title

Build an ETL pipeline in airflow

Describe your Talk

In this talk I’ll be giving a walk through on how to make a basic ETL pipeline using airflow

Apache Airflow is an open-source tool that helps automate, schedule, and manage complex workflows built by Airbnb.

It uses Python to define workflows, allowing users to orchestrate tasks, handle dependencies, and monitor data flow across systems efficiently.

It's widely used for managing data pipelines and automating processes.

Developers can build workflows that are divided into small tasks (or nodes) and organized into Directed Acyclic Graphs (DAGs).

These DAGs are powerful as they allow you to visually check each step of the pipeline, view logs for each node, retry on failure and more.

Airflow web ui and a DAG

Pre-requisites & reading material

Basic Python is enough

Time required for the talk

30-45min

Link to slides/demos

To be updated

About you

I’m a student with about 2 years of experience as a fullstack web developer. I’ve made react apps, fastAPI backends and even deployed an Airflow ETL pipeline in production. I enjoy writing python, typescript and rust.

Availability

16/12/2023

Any comments

No response

[pulsar17]

Hi @Maniktherana , will you be available to give the talk on January 20, 2024 (3rd Saturday)?

[Maniktherana]

Hey @pulsar17, yes I should be available

[pulsar17]

@Maniktherana your talk is scheduled on 20th! You can find the venue and timings in https://www.meetup.com/pydelhi/events/298483033/

[pulsar17]

@Maniktherana please either share your Telegram username or DM me on Telegram so that there is a way to contact you tomorrow. (My Telegram username is @pulsar17)

[Maniktherana]

@pulsar17 messaged you

[pulsar17]

Thanks for giving a talk on this @Maniktherana ! Closing this issue now.