search

openradar / erad2024

Repo for the ERAD 2024 Open Radar Short Course
https://openradarscience.org/erad2024/
Apache License 2.0
5 stars 6 forks source link

Add reader for italian radar data data provided by Gianfranco Vulpiano #7

Open wolfidan opened 2 months ago

wolfidan commented 2 months ago

Hi, Jordi asked Gianfranco Vulpiani for a usable dataset for the ERAD short course and he provided this:

https://zenodo.org/records/4897697

The data uses a proprietary format called Datamet (https://zenodo.org/records/4897245). We would need to write a reader to ingest this data for the course. I can handle that. Should I write it in xradar or pyart?

Thanks, Cheers, Daniel

kmuehlbauer commented 2 months ago

@wolfidan xradar would be preferred. :grinning:

kmuehlbauer commented 1 week ago

@wolfidan If you need some assistance with your current code, please let me know

wolfidan commented 1 week ago

Thank you @kmuehlbauer , I'm on it now, I'll keep you posted.

wolfidan commented 1 week ago

Hi @kmuehlbauer , @mgrover1

I made a first attempt here: https://github.com/wolfidan/xradar/blob/main/xradar/io/backends/datamet.py

However I am not sure how to proceed to make it available in xarray?

The first part of this code works (up to xr.open_dataset):

f = '/scratch/wolfensb/erad2024/datamet_reader/H-000-VOL-ILMONTE-201907100700.tar'
import xarray as xr
import xradar as xd
entry = xd.io.backends.DataMetEntrypoint()
ds = entry.open_dataset(f, group='sweep_1')
xr.open_dataset(f, group='sweep_1', engine = 'datamet')

But the xarray open_dataset fails with

ValueError: unrecognized engine datamet must be one of: ['netcdf4', 'h5netcdf', 'scipy', 'cfgrib', 'cfradial1', 'furuno', 'gamic', 'gini', 'iris', 'nexradlevel2', 'odim', 'radolan', 'rainbow', 'store', 'wradlib-cfradial1', 'wradlib-cfradial2', 'wradlib-furuno', 'wradlib-gamic', 'wradlib-iris', 'wradlib-odim', 'wradlib-rainbow']

Is there something else I need to do to make it available? Note: xradar (with my modifs) was installed with python -m pip install .

(I admit I didn't look too much in detail about how xarray gets the backends from xradar)

Thanks a lot

kmuehlbauer commented 1 week ago

@wolfidan

Great!

This should work, too, for your testing:

xr.open_dataset(f, group='sweep_1', engine=xd.io.backends.DataMetEntrypoint)
kmuehlbauer commented 1 week ago

@wolfidan If we are allowed to put one testfile on open-radar-data then we could use that for testing and example notebook. If you think this is ready for a PR please go for it.

wolfidan commented 1 week ago

@kmuehlbauer Data is publicly available so I think it is ok to upload one file. Thanks for your help ! Not yet ready for PR, I need to write tests and do additional checks, but I'll try to make it quick