Make cfradial generic - Githubissues

mgrover1 commented 1 year ago

Closes #42

mgrover1 commented 1 year ago

Good question - I can rerun black on these changes

kmuehlbauer commented 1 year ago

Another question would be, why does black not complain about?

kmuehlbauer commented 1 year ago

There is this

https://github.com/openradar/xradar/blob/main/pyproject.toml#L17

mgrover1 commented 1 year ago

@kmuehlbauer this should be good to go

kmuehlbauer commented 1 year ago

Great, this looks much better now :-) I'll have a thorough review the next day.

mgrover1 commented 1 year ago

Great! I think the one catch here is that we should strip out the coordinates stored at the root level...

kmuehlbauer commented 1 year ago

Yes, we should already take much care to move variables and attributes to those groups, where the standard expects them. The example data in the cfradial1 notebook is a good exercise to check this.

So the r_calib most likely need to be moved into a separate calibration group. There are other variables/attributes which need to be stripped/moved too. Best we be now as exact as possible with that procedure.

kmuehlbauer commented 1 year ago

@mgrover1 To be on the same page, this is the plain output of the CfRadial1 dataset:

<xarray.Dataset>
Dimensions:                           (sweep: 9, r_calib: 1, time: 4343,
                                       range: 996)
Coordinates:
  * time                              (time) datetime64[ns] 2008-06-04T00:15:...
  * range                             (range) float32 150.0 300.0 ... 1.494e+05
Dimensions without coordinates: sweep, r_calib
Data variables:
    volume_number                     int32 ...
    platform_type                     |S32 ...
    primary_axis                      |S32 ...
    status_xml                        |S1 ...
    instrument_type                   |S32 ...
    radar_antenna_gain_h              float32 ...
    radar_antenna_gain_v              float32 ...
    radar_beam_width_h                float32 ...
    radar_beam_width_v                float32 ...
    radar_rx_bandwidth                float32 ...
    time_coverage_start               |S32 ...
    time_coverage_end                 |S32 ...
    azimuth_correction                float32 ...
    elevation_correction              float32 ...
    range_correction                  float32 ...
    longitude_correction              float32 ...
    latitude_correction               float32 ...
    pressure_altitude_correction      float32 ...
    altitude_correction               float32 ...
    eastward_velocity_correction      float32 ...
    northward_velocity_correction     float32 ...
    vertical_velocity_correction      float32 ...
    heading_correction                float32 ...
    roll_correction                   float32 ...
    pitch_correction                  float32 ...
    drift_correction                  float32 ...
    rotation_correction               float32 ...
    tilt_correction                   float32 ...
    latitude                          float64 ...
    longitude                         float64 ...
    altitude                          float64 ...
    sweep_number                      (sweep) int32 ...
    sweep_mode                        (sweep) |S32 ...
    polarization_mode                 (sweep) |S32 ...
    prt_mode                          (sweep) |S32 ...
    follow_mode                       (sweep) |S32 ...
    fixed_angle                       (sweep) float32 ...
    target_scan_rate                  (sweep) float32 ...
    sweep_start_ray_index             (sweep) int32 ...
    sweep_end_ray_index               (sweep) int32 ...
    r_calib_time                      (r_calib) |S32 ...
    r_calib_pulse_width               (r_calib) timedelta64[ns] ...
    r_calib_xmit_power_h              (r_calib) float32 ...
    r_calib_xmit_power_v              (r_calib) float32 ...
    r_calib_two_way_waveguide_loss_h  (r_calib) float32 ...
    r_calib_two_way_waveguide_loss_v  (r_calib) float32 ...
    r_calib_two_way_radome_loss_h     (r_calib) float32 ...
    r_calib_two_way_radome_loss_v     (r_calib) float32 ...
    r_calib_receiver_mismatch_loss    (r_calib) float32 ...
    r_calib_radar_constant_h          (r_calib) float32 ...
    r_calib_radar_constant_v          (r_calib) float32 ...
    r_calib_antenna_gain_h            (r_calib) float32 ...
    r_calib_antenna_gain_v            (r_calib) float32 ...
    r_calib_noise_hc                  (r_calib) float32 ...
    r_calib_noise_vc                  (r_calib) float32 ...
    r_calib_noise_hx                  (r_calib) float32 ...
    r_calib_noise_vx                  (r_calib) float32 ...
    r_calib_receiver_gain_hc          (r_calib) float32 ...
    r_calib_receiver_gain_vc          (r_calib) float32 ...
    r_calib_receiver_gain_hx          (r_calib) float32 ...
    r_calib_receiver_gain_vx          (r_calib) float32 ...
    r_calib_base_dbz_1km_hc           (r_calib) float32 ...
    r_calib_base_dbz_1km_vc           (r_calib) float32 ...
    r_calib_base_dbz_1km_hx           (r_calib) float32 ...
    r_calib_base_dbz_1km_vx           (r_calib) float32 ...
    r_calib_sun_power_hc              (r_calib) float32 ...
    r_calib_sun_power_vc              (r_calib) float32 ...
    r_calib_sun_power_hx              (r_calib) float32 ...
    r_calib_sun_power_vx              (r_calib) float32 ...
    r_calib_noise_source_power_h      (r_calib) float32 ...
    r_calib_noise_source_power_v      (r_calib) float32 ...
    r_calib_power_measure_loss_h      (r_calib) float32 ...
    r_calib_power_measure_loss_v      (r_calib) float32 ...
    r_calib_coupler_forward_loss_h    (r_calib) float32 ...
    r_calib_coupler_forward_loss_v    (r_calib) float32 ...
    r_calib_zdr_correction            (r_calib) float32 ...
    r_calib_ldr_correction_h          (r_calib) float32 ...
    r_calib_ldr_correction_v          (r_calib) float32 ...
    r_calib_system_phidp              (r_calib) float32 ...
    r_calib_test_power_h              (r_calib) float32 ...
    r_calib_test_power_v              (r_calib) float32 ...
    r_calib_receiver_slope_hc         (r_calib) float32 ...
    r_calib_receiver_slope_vc         (r_calib) float32 ...
    r_calib_receiver_slope_hx         (r_calib) float32 ...
    r_calib_receiver_slope_vx         (r_calib) float32 ...
    azimuth                           (time) float32 ...
    elevation                         (time) float32 ...
    pulse_width                       (time) timedelta64[ns] ...
    prt                               (time) timedelta64[ns] ...
    prt_ratio                         (time) timedelta64[ns] ...
    nyquist_velocity                  (time) float32 ...
    unambiguous_range                 (time) float32 ...
    antenna_transition                (time) int8 ...
    n_samples                         (time) int32 ...
    r_calib_index                     (time) int8 ...
    measured_transmit_power_h         (time) float32 ...
    measured_transmit_power_v         (time) float32 ...
    scan_rate                         (time) float32 ...
    DBZ                               (time, range) float32 ...
    VR                                (time, range) float32 ...
Attributes:
    Conventions:         CF/Radial instrument_parameters radar_parameters rad...
    version:             1.2
    title:               TIMREX
    institution:         
    references:          
    source:              
    history:             
    comment:             
    instrument_name:     SPOLRVP8
    site_name:           
    scan_name:           
    scan_id:             0
    platform_is_mobile:  false
    n_gates_vary:        false

Here are my thoughts on data variables:

data variables with time or time/range dimensions -> respective sweep group
data variables with sweep dimension -> respective sweep group (or removed if not needed)
data variables with r_calib dimensions -> calibration group
other variables (eg. radar_antenna_gain_h, eastward_velocity_correction) -> root group or other sub-group???

mgrover1 commented 1 year ago

@kmuehlbauer - I agree with your description here. Unfortunately I will not be able to work on this today, but agree.

kmuehlbauer commented 1 year ago

@mgrover1 No worries, I might play with this a bit the next days. If I come up with something substantial I'll share it here.

kmuehlbauer commented 1 year ago

@mgrover1 I've taken a stab at this. Had to refactor substantially. I'll have a example notebook on how to map the CfRadial1 model too the CfRadial2 model. Put this in another PR next week.

kmuehlbauer commented 1 year ago

Extended in #45

openradar / xradar

Make cfradial generic #43