Update the satellite bias coefficient(BC) upgrader following naming conventions

huishao-r commented 3 months ago

jedi-ci-build-cache=skip

Description

During the earlier VarBC code sprint, we decided to update the BC files with new format as well as following the IODA naming conventions. This upgrader was initially developed by Cory updates the BC files for the dimension part as well as some variable name changes. This PR is intended to complete this work, following the discussions with our kinds teams (EMC, MetOffice).

Please note, this upgrader only works for the BC file in the format being used currently in develop (ufo-data)! The data files in VarBC-sprint (which has partial changes already) shouldnt use the upgrader in this PR.

The new format will have two-dimension (Channel and Record) arrays for coefficients and errors per predictor. For satellite radiance BC, the dimension Record is not being used at this point and therefore is fixed to be 1. This Record dimension will be adopted in the new future for aircraft BC (or BC for other observation types.). For now, this PR focuses on satellite radiance (or any channel dependent observations).

Naming convention: (1) uses CamelCase, instead of underscore in the variable names, unless some exceptions, e.g. scan_angle_order_1-> sensorScanAngle_order_1 (2) uses existing variable names from observation naming convention: e.g., channels->sensorChannelNumber, cloud_liquid_water->cloudWaterContent

Use of this upgrader: python3 satbias_upgrader.py -i input.nc4 -o output.nc4

Impact

The commit this upgrader wouldn't have any impacts. However, use of this upgrader to BC files in develop would have impacts on coding and tests.

Checklist

[x] I have performed a self-review of my own code
[ ] I have made corresponding changes to the documentation
[ ] I have run the unit tests before creating the PR

PatNichols commented 3 months ago

@huishao-r Coding norms error. I will fix it later today.

huishao-r commented 3 months ago

@PatNichols You need to merge develop to feature/VarBC-sprint first? Then merge feature/VarBC-sprint to this branch. Otherwise, we will continue to see additional files (from develop) diverges from feature/VarBC-sprint.

BenjaminRuston commented 3 months ago

@huishao-r if you are happy with this you should be able to merge it since it's not to develop and into a branch

JCSDA-internal / ioda-converters