Allow arbitrary fields to be read into the cfradial1 reader

[mgrover1]

• xradar version: 0.0.8
• Python version: 3.10
• Operating System: MacOS

Description

I am trying to read in a file with non-standard names for variables (ex. KDP). If these fields are in the underlying netCDF file, we should pass them into the object. For now, it is silently subsetting the variables in the configuration/data model module.

What I Did

ds = xd.io.open_cfradial1_datatree('test-pol.nc', first_dim='auto')
DataTree('root', parent=None)
│   Dimensions:              ()
│   Data variables:
│       latitude             float64 38.9
│       longitude            float64 -106.9
│       altitude             float64 3.149e+03
│       time_coverage_start  |S192 b'2022-08-25T19:46:43Z'
│       time_coverage_end    |S192 b'2022-08-25T19:47:14Z'
│       volume_number        int32 218053
│       platform_type        |S192 b'fixed'
│       instrument_type      |S192 b'radar'
│   Attributes:
│       Conventions:      CF/Radial
│       title:            weather_radar_volume
│       instrument_name:  csu.sail
│       history:          created by jrobrien on jupyter02.mcp.alcf.anl.gov at 20...
└── DataTree('sweep_0')
        Dimensions:       (azimuth: 1154, range: 675)
        Coordinates:
            time          (azimuth) datetime64[ns] 2022-08-25T19:47:14 ... 2022-08-25...
          * range         (range) float32 -118.4 -58.48 1.46 ... 4.022e+04 4.028e+04
          * azimuth       (azimuth) float32 0.002944 0.07436 0.1293 ... 359.8 359.9
            elevation     (azimuth) float32 1.011 1.011 1.011 ... 1.011 1.011 1.011
            latitude      float64 38.9
            longitude     float64 -106.9
            altitude      float64 3.149e+03
        Data variables:
            DBZ           (azimuth, range) float64 ...
            VEL           (azimuth, range) float64 ...
            ZDR           (azimuth, range) float64 ...
            PHIDP         (azimuth, range) float64 ...
            RHOHV         (azimuth, range) float64 ...
            NCP           (azimuth, range) float64 ...
            SNR           (azimuth, range) float64 ...
            sweep_number  int32 0
            fixed_angle   float32 1.011
            sweep_mode    |S192 b'azimuth_surveillance\x00\x00\x00\x00azimuth_surveil...

Despite there being more fields in the file, which upon using the regular netCDF4 backend, results in

<xarray.Dataset>
Dimensions:                      (time: 1154, range: 675, sweep: 1)
Coordinates:
  * time                         (time) datetime64[ns] 2022-08-25T19:46:43 .....
  * range                        (range) float32 -118.4 -58.48 ... 4.028e+04
    azimuth                      (time) float32 ...
    elevation                    (time) float32 ...
Dimensions without coordinates: sweep
Data variables: (12/31)
    DBZ                          (time, range) float64 ...
    VEL                          (time, range) float64 ...
    WIDTH                        (time, range) float64 ...
    ZDR                          (time, range) float64 ...
    PHIDP                        (time, range) float64 ...
    RHOHV                        (time, range) float64 ...
    ...                           ...
    time_coverage_end            |S192 ...
    time_reference               |S192 ...
    volume_number                int32 ...
    platform_type                |S192 ...
    instrument_type              |S192 ...
    primary_axis                 |S192 ...
Attributes:
    Conventions:      CF/Radial
    title:            weather_radar_volume
    instrument_name:  csu.sail
    field_names:      DBZ, VEL, WIDTH, ZDR, PHIDP, RHOHV, NCP, SNR, DBZhv
    history:          created by jrobrien on jupyter02.mcp.alcf.anl.gov at 20...

[kmuehlbauer]

KDP will be catched, but here WIDTH and DBZhv isn't covered. Could you make the file available someplace? I think we only need to add these (two) names to

https://github.com/openradar/xradar/blob/28ab80676986660107d4eb70eef26b2090fe7a09/xradar/model.py#L207-L211

[mgrover1]

@kmuehlbauer - can we make this generic where we do not automatically filter fields?

[kmuehlbauer]

Would not consider that as bug, but a naming issue.

Several options:

1. allow only standard
2. allow standard, provide allowed non_standard (as is now)
3. allow standard and anything else

If we decide on 3, we'd need a function which checks for standard naming convention.

[kmuehlbauer]

Another idea would be to let the user provide a mapping ala {"myKDP": "KDP"} when reading the file.

[mgrover1]

Let's discuss this at today's meeting :)

[kmuehlbauer]

Unfortunately that's already too late in the evening :zzz:

[mgrover1]

Okay - I am leaning toward the third approach... I think it is important that the object in memory is as similar to the file on disk as possible...

[kmuehlbauer]

I'm good with that, but at least we would have to make sure that we only write standard conforming files. In those cases we would need to alert the user about standard conformance.

[mgrover1]

@kmuehlbauer agreed! We could return a warning that certain won't write, and suggest they conform to the standard.

[kmuehlbauer]

solved by #45