ICON-D2: Demo pipeline for acquisition, area of interest extraction and plotting

[amotl]

Dear Michael,

based on your improvements to the dwd-open-data-downloader by @EduardRosert and @bjoern-reetz you published at https://github.com/EduardRosert/docker-dwd-open-data-downloader/pull/2 (thanks a stack to all of you!), I've just added a demo of a humble pipeline machinery [1] which might resonate with your proposal at https://github.com/earthobservations/wetterdienst/discussions/271#discussioncomment-224863.

The goal is to have as much details available as Python code as possible. So, one detail that was important to me was to have the "download/processing recipe" available as Python code in order to lay the groundwork of evolving this into different directions, see [2].

It is really only a demo, completely unpolished yet and it might not even work on Windows. It has to be invoked from the toplevel directory. A short synopsis about installation and operation would be:

# Acquire sources
git clone https://github.com/earthobservations/gribmagic
cd gribmagic

# Install ECMWF ecCodes, Climate Data Operators (CDO) and ECMWF Magics. YMMV. See also `docs/magics.rst`.
# See also https://confluence.ecmwf.int/display/ECC and https://code.mpimet.mpg.de/projects/cdo.

# Linux
apt-get install --yes libeccodes0 cdo libmagplus3v5

# macOS
brew install eccodes cdo
make magics-install
make magics-info

# Python virtualenv FTW
python3 -m venv .venv
source .venv/bin/activate

# Install Python package
pip install --editable=.[plotting]

# Install improved DWD Open Data Downloader
gribmagic install dwd-grib-downloader

# Invoke the machinery
./demo/acquire_and_bbox.sh

While still in its infancy, I believed it would be a good idea to share this with you and will be happy about any feedback.

With kind regards, Andreas.

[1] https://github.com/earthobservations/gribmagic/blob/8d5ffdc/demo/pipeline.sh [2] https://github.com/earthobservations/gribmagic/blob/8d5ffdc/demo/pipeline/recipe.py

cc @gutzbenj, @wetterfrosch, @mhaberler

---

Demo plot

-- .gribmagic-data/subgrid/bbox_46.0_47.5_14.5_16.8/png/icon-d2_germany_regular-lat-lon_pressure-level_2020122212_001_950_v.grib2.png

[mhaberler]

Hi @amotl

sorry for the confusion - I switched to master in https://github.com/mhaberler/docker-dwd-open-data-downloader/commits/master which contains the latest and greatest (and it will stay there! )

will give this a spin - currently investigating how to add radiosonde flights to my animations

potential sources:

habhub.org data, accessible from: receivers: http://spacenear.us/tracker/receivers.php predictions: http://spacenear.us/tracker/get_predictions.php

example I am working on: curl --max-time 600 --output positions.json 'https://spacenear.us/tracker/datanew.php?mode=6hours&type=positions&format=json&max_positions=0'

other potential sources: https://radiosondy.info/ https://opendata.dwd.de/weather/weather_reports/radiosonde/trajectories/ in json and kml format https://www.ncdc.noaa.gov/data-access/weather-balloon/integrated-global-radiosonde-archive

here's a first stab: https://cesium.com/cesiumjs/cesium-viewer/?source=https://static.mah.priv.at/cors/habhub.czml

[mhaberler]

oh wow, that is VERY cool!!