NOAA-EMC / GDASApp

Global Data Assimilation System Application
GNU Lesser General Public License v2.1
14 stars 28 forks source link

Add ability for JEDI-to-FV3 increment converter to process ensembles #1022

Closed DavidNew-NOAA closed 3 months ago

DavidNew-NOAA commented 3 months ago

Major changes to JEDI-to-FV3 increment converter:

  1. Allow processing of ensembles using the same YAML template format that the FV3-JEDI LGETKF app uses. Alternatively, a list of background/increments can be specified.

  2. Fix incorrect calculation of FV3 increment, for variables other than delp and hydrostatic_delz, related to two issues: a. When adding the JEDI increment to the background to get the analysis, some mixing-ratio-type variables become negative and are set to zero by FV3-JEDI. However, when we then subtract the analysis and background after the variable change to get delp and hydrostatic_delz, we don't get the same increment that we started with for those mixing-ratio fields. b. For ensemble DA, the deterministic background and JEDI increment are not the same resolution, so the background and analysis discussed above must be interpolated to half resolution before being subtracted but after the JEDI increment is interpolated to full resolution and added to the background. Because of these two interpolations, similar to above, we don't get the same increment as we started with.

I now use Atlas to manually set all variables other than delp and hydrostatic_delz back to their original values, which solved both issues.

Additionally, I make the following other changes:

  1. Rename fv3jedi_fv3inc.yaml to fv3jedi_fv3inc_variational.yaml and rewrite according to the new YAML format.
  2. Rewrite the gdasapp_test_fv3jedi_fv3inc CTest input YAML according to the new YAML format.
  3. Create a new fv3jedi_fv3inc_lgetkf.yaml for an eventual use in converting the ensemble job increment in Global Workflow.

Testing these new features yields the following results:

  1. For delp and hydrostatic_delz in ensemble DA, the FV3 increment produced by the deterministic variational analysis in Global Workflow (branch feature/jediinc2fv3) is zero diff with develop, which has the previous version of the OOPS-based increment converter. I tested the other increment variables by writing the FV3 increment out to the native cubed sphere and compared with the JEDI increment. They are zero diff.
  2. The gdasapp_test_fv3jedi_fv3inc passes with the same test reference used in develop.
  3. Using a special Global Workflow branch to test the lgetkf analysis, the YAML templating works, and the resulting FV3 increment variables, delp and hydrostatic_delz, differ from the one values produced by develop (using jediinc2fv3.py) within a reasonable relative error, similar to that when I first tested the increment converter against jediinc2fv3.py with the variational analysis.
DavidNew-NOAA commented 3 months ago

Actually I have one more idea for a testing strategy, then will re-open...

DavidNew-NOAA commented 3 months ago

After a ton of debugging and testing, I'm finally re-opening this PR