search

CityofPittsburgh / data-rivers

Apache Airflow and Beam ETL scripts for the City of Pittsburgh's data analysis pipelines
10 stars 1 forks source link

creation of dags to run the entire pipelines for the pli permits. cur… #764

Closed woodjt closed 2 months ago

woodjt commented 2 months ago

…rently the only stage in the dag is the extraction and all others will be added as they are created. this allows us to run the extraction stage in the dag in drt airflow for testing