Overview:
This pull request adds a new DAG (fetch_cdc_air_temp_DAG) to the ETL pipeline - the workflow for this DAG is a distinct replica of the last PR raised ( #9 ), only difference here is that the data dealt here is centered on Air temperature mean of some sample lat/long locations.
Changes Made:
Added a new DAG script fetch_cdc_air_temp_DAG.py that fetches and clips the air temp data from the provided FTP site
Updated the dag_ids list on the primary_DAG.py script to include the newly added CDC DAG (fetch_cdc_air_temp) as part of the downstream DAGs to trigger
Overview: This pull request adds a new DAG (
fetch_cdc_air_temp_DAG) to the ETL pipeline - the workflow for this DAG is a distinct replica of the last PR raised ( #9 ), only difference here is that the data dealt here is centered on Air temperature mean of some sample lat/long locations.Changes Made:
fetch_cdc_air_temp_DAG.pythat fetches and clips the air temp data from the provided FTP sitedag_idslist on theprimary_DAG.pyscript to include the newly added CDC DAG (fetch_cdc_air_temp) as part of the downstream DAGs to trigger