openradar / xradar

A tool to work in weather radar data in xarray
https://docs.openradarscience.org/projects/xradar
MIT License
85 stars 17 forks source link

Add DataMet backend for Italian Radar data #175

Open wolfidan opened 1 week ago

wolfidan commented 1 week ago

Dear xradar people, this is a first try at adding a new backend to read the data from the Italian radars provided by Gianfranco Vulpiani that we plan to use at the open radar course at ERAD2024.

https://zenodo.org/records/4897697

I must say that I had some trouble understanding all the classes that are required to wrap it into xarray, but I tried my best and it seems to work, as I was able to read all 165 provided files. I have added some tests but they are somewhat simplistic at the moment and only test the basic reader not the wrapper to xarray (other tests seem to do the same things).

The DataMet files are not consistent in which sweep corresponds to which elevation: sometimes sweep 1 could be 16° other times 90°. I find it a bit weird but left it as is and didn't try to remap it. I wonder if this could be an issue when reading them into pyart?

Thank you for your time. Next week I will be in vacation and might not answer immediately, sorry about that.

kmuehlbauer commented 6 days ago

Thanks @wolfidan! The ruff and numpy2 issues should be resolved now by #177. I'll have a closer look here later.

codecov[bot] commented 6 days ago

Codecov Report

Attention: Patch coverage is 54.37788% with 99 lines in your changes missing coverage. Please review.

Project coverage is 89.66%. Comparing base (89d9a3e) to head (883bc09).

Files Patch % Lines
xradar/io/backends/datamet.py 54.16% 99 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #175 +/- ## ========================================== - Coverage 91.58% 89.66% -1.92% ========================================== Files 21 22 +1 Lines 3991 4208 +217 ========================================== + Hits 3655 3773 +118 - Misses 336 435 +99 ``` | [Flag](https://app.codecov.io/gh/openradar/xradar/pull/175/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openradar) | Coverage Δ | | |---|---|---| | [notebooktests](https://app.codecov.io/gh/openradar/xradar/pull/175/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openradar) | `77.54% <24.42%> (-2.89%)` | :arrow_down: | | [unittests](https://app.codecov.io/gh/openradar/xradar/pull/175/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openradar) | `87.78% <54.37%> (-1.82%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openradar#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

kmuehlbauer commented 6 days ago

@wolfidan I've taken the chance to fix this up to green CI. Looks good so far. We might just copy/adapt code/tests from the other backends to finish this.

Thanks for this nice addition!

kmuehlbauer commented 6 days ago

This works for me with different elevations. Looks that this is a top down strategy.

import open_radar_data
import xarray as xr
fname = open_radar_data.DATASETS.fetch("H-000-VOL-ILMONTE-201907100700.tar.gz")
with xr.open_dataset(fname, engine="datamet", group="sweep_8") as ds:
    display(ds)
    ds = ds.set_coords("sweep_mode")
    ds = ds.wrl.georef.georeference()
    ds.DBTH.wrl.vis.plot()
kmuehlbauer commented 6 days ago

The DataMet files are not consistent in which sweep corresponds to which elevation: sometimes sweep 1 could be 16° other times 90°.

Maybe it's that the 90deg sweep is not measured in every volume?

wolfidan commented 5 days ago

That's amazing, thanks a lot @kmuehlbauer, I will add more tests the week after the next to increase codecov