breedfides / airflow-etl

0 stars 2 forks source link

Implement air temperature data fetching DAG (DWD CDC netCDF) #6

Closed gannebamm closed 8 months ago

gannebamm commented 11 months ago

Expected behaviour

There are ready-to-use DAGs for the second environmental dataset. This DAG shall extract data from DWD CDC:

air temperature

metadata: https://opendata.dwd.de/climate_environment/CDC/grids_germany/daily/Project_TRY/air_temperature_mean/DESCRIPTION_gridsgermany_daily_Project_TRY_air_temperature_mean_en.pdf

files: https://opendata.dwd.de/climate_environment/CDC/grids_germany/daily/Project_TRY

The datasets are stored at the CDC FTP and in netCDF format. The netCDFs are zipped. The radiation data from 2000 onwards shall be downloaded, the zip extracted and the netCDF clipped to the given lat / long position + 3km radius.