breedfides / airflow-etl

0 stars 2 forks source link

Buffer distance ≤ 1000 m yields empty output for DWD data #16

Closed vineetasharma105 closed 6 months ago

vineetasharma105 commented 6 months ago

The spatial resolution of DWD data (1. global radiation and 2. air_temperature_mean) is 1 km x 1 km. (check links : https://opendata.dwd.de/climate_environment/CDC/grids_germany/daily/Project_TRY/air_temperature_mean/DESCRIPTION_gridsgermany_daily_Project_TRY_air_temperature_mean_en.pdf and https://opendata.dwd.de/climate_environment/CDC/grids_germany/daily/Project_TRY/radiation_global/DESCRIPTION_gridsgermany_daily_Project_TRY_radiation_global_en.pdf)

It has been found that due to the spatial resolution of the underlying DWD datasets, the clipping extent should be more than 1 km in order to generate a valid output.

@gannebamm

gannebamm commented 6 months ago

A user could use a small buffer value since the buffer in meters input is a parameter. As default, 3000m is used as stated here: https://github.com/breedfides/airflow-etl/blob/main/dags/src/utility.py#L243

This will generate valid output. I do not see the need to change this default for now. Nonetheless, with the work of https://github.com/breedfides/airflow-etl/pull/12#pullrequestreview-1830076007, the frontend team could create a webform which changes the buffer according to user input. If this is needed, we shall present a warning there.