search

openradar / xradar

A tool to work in weather radar data in xarray
https://docs.openradarscience.org/projects/xradar
MIT License
88 stars 17 forks source link

ENH: Generalize CfRadial1 reader #45

Closed kmuehlbauer closed 1 year ago

kmuehlbauer commented 1 year ago

This is an attempt to increase the amount of metadata which is imported into the internal DataTree-structure at the right places CfRadial2 is expecting it.

Takes over from: #43

ToDo:

kmuehlbauer commented 1 year ago

@mgrover1 I've squashed your commits from #43 into one and added my changes on top of it.

The initial implementation wasn't that bad already :-)

codecov[bot] commented 1 year ago

Codecov Report

Merging #45 (2cd6d9e) into main (596ce94) will increase coverage by 0.11%. The diff coverage is 93.18%.

@@            Coverage Diff             @@
##             main      #45      +/-   ##
==========================================
+ Coverage   85.49%   85.61%   +0.11%     
==========================================
  Files          15       15              
  Lines        3020     3079      +59     
==========================================
+ Hits         2582     2636      +54     
- Misses        438      443       +5
Flag Coverage Δ
unittests 85.61% <93.18%> (+0.11%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
xradar/georeference/transforms.py 100.00% <ø> (ø)
xradar/io/backends/gamic.py 86.61% <ø> (ø)
xradar/io/backends/iris.py 81.61% <ø> (ø)
xradar/io/backends/odim.py 87.98% <ø> (ø)
xradar/io/backends/rainbow.py 82.80% <ø> (ø)
xradar/io/backends/cfradial1.py 85.90% <92.68%> (+3.68%) :arrow_up:
xradar/io/export.py 95.04% <100.00%> (ø)
xradar/model.py 94.15% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

kmuehlbauer commented 1 year ago

@mgrover1 It would be great, if you could test this PR against some of your CfRadial1 files if you find the time.

Dual frequency cases would be great to have and also sweep subgroup cases would be interesting (also for testing). So if anyone has some of these special case CfRadial1 data files available for open sharing, that would be great.

mgrover1 commented 1 year ago

This looks great @kmuehlbauer - I tried it on the sample dataset, and it worked well

print(ds['sweep_0'])

DataTree('sweep_0', parent="root")
    Dimensions:                      (azimuth: 1154, range: 675)
    Coordinates:
        time                         (azimuth) datetime64[ns] 2022-08-25T19:47:14...
      * range                        (range) float32 -118.4 -58.48 ... 4.028e+04
      * azimuth                      (azimuth) float32 0.002944 0.07436 ... 359.9
        elevation                    (azimuth) float32 1.011 1.011 ... 1.011 1.011
        latitude                     float64 38.9
        longitude                    float64 -106.9
        altitude                     float64 3.149e+03
    Data variables: (12/28)
        DBZ                          (azimuth, range) float64 ...
        VEL                          (azimuth, range) float64 ...
        WIDTH                        (azimuth, range) float64 ...
        ZDR                          (azimuth, range) float64 ...
        PHIDP                        (azimuth, range) float64 ...
        RHOHV                        (azimuth, range) float64 ...
        ...                           ...
        time_coverage_end            |S192 b'2022-08-25T19:47:14Z'
        time_reference               |S192 b'1970-01-01T00:00:00Z'
        volume_number                int32 218053
        platform_type                |S192 b'fixed'
        instrument_type              |S192 b'radar'
        primary_axis                 |S192 b'axis_z'
kmuehlbauer commented 1 year ago

@mgrover1 This looks like the version you run is not the correct version. I've found that installing xradar via pip sometimes keeps old versions (and uses them). I did not come behind that. I'm removing those by hand from site-packages-folder.

In your case the latitude/longitude/altitude should not be in the sweep-Dataset. They can be found in the root-group.

If the user reads a complete volume as DataTree they will get site-coords with the root-group. But if they open a single sweep with xr.open_dataset they will not get the site-coords with the sweep. I'm wondering if we should allow site-coordinates in the sweep-Dataset when loaded with the xarray-Backend? Or would that confuse users, if they don't have the site-coords in the sweep-groups (DataTree) but have them in the sweep-Datasets (xr.open_dataset)?

If this is something we can defer to later discussion, I'm good with merging this. I'll open follow-up issues for the todo's listed above.

kmuehlbauer commented 1 year ago

Here is sweep_0-group with the correct reader with this PR.

DataTree('sweep_0', parent="root")
    Dimensions:                    (time: 483, range: 996)
    Coordinates:
      * time                       (time) datetime64[ns] 2008-06-04T00:15:03 ... ...
      * range                      (range) float32 150.0 300.0 ... 1.494e+05
        azimuth                    (time) float32 ...
        elevation                  (time) float32 ...
    Data variables: (12/18)
        sweep_number               int32 ...
        sweep_mode                 |S32 ...
        prt_mode                   |S32 ...
        follow_mode                |S32 ...
        sweep_fixed_angle          float32 ...
        pulse_width                (time) timedelta64[ns] ...
        ...                         ...
        r_calib_index              (time) int8 ...
        measured_transmit_power_h  (time) float32 ...
        measured_transmit_power_v  (time) float32 ...
        scan_rate                  (time) float32 ...
        DBZ                        (time, range) float32 ...
        VR                         (time, range) float32 ...
kmuehlbauer commented 1 year ago

OK, so hopefully everyone is aware of the above minor issues. Pulling this in.