Update feature/gdas-validation to be consistent with g-w and JEDI changes

[RussTreadon-NOAA]

g-w and JEDI have been updated and this requires corresponding updates in feature/gdas-validation. This PR adds the required updates to feature/gdas-validation. Changes are also made which allow GDAS validation to run on Hera in addition to Orion.

Fixes #105

GSI and JEDI GDAS validation were run on Orion and Hera for 2021080100. GSI prep, anal, and analdiag jobs ran to completion. JEDI prepatmiodaobs, atmanlinit, atmanlrun, and atmanlfinal ran to completion.

JEDI testing was done with the following local changes:

• edit parm/atm/obs/lists/gdas_prototype_3d.yaml to process atms_n20, satwind_abi_goes-16, scatwind_ascat_metop-a, and gnssro
• move ush/ioda/bufr2ioda/bufr2ioda_subpfl_argo_profiles.py to ush/ioda/bufr2ioda/skip_bufr2ioda_subpfl_argo_profiles.py to prevent this script from being executed by run_bufr2ioda.py. The bufr dump file processed by bufr2ioda_subpfl_argo_profiles.py does not exist in the DMPDIR for atmospheric observations. Hence bufr2ioda_subpfl_argo_profiles.py aborts. The proposed solution is to change run_bufr2ioda.py so that it reads a list of observation dump files to process instead of running all scripts which begin with bufr2ioda. This change will be in a future GDASApp PR.
• set CASE_ANL=C384 in config.atmanl. Using C768 results in fv3jedi_var.x aborting with the error message stripack_addnod: colinear points
• NOTE: atmanlfinal only tars diagnostic files ending in suffix .nc4. Some GDASApp observation yamls use .nc while others use .nc4. Unidata recommends .nc (see What convention should be used for the names of netCDF files?). Adhering to this recommendation will require future PRs to change g-w atm_analysis.py and GDASApp observation yamls.

[emilyhcliu]

@RussTreadon-NOAA Thanks a lot for working on this. I am testing now. Will come back to report my results later.

[RussTreadon-NOAA]

Thank you @emilyhcliu . If your results are acceptable, please approve this PR. I will then merge it into the authoritative feature/gdas-validation.

[emilyhcliu]

@RussTreadon-NOAA I checked feature/gdas-validation from your fork and tested three steps for setup_Workspace.sh:

1. clone --- successful
2. build --- successful
3. setup --- successful

Tested three JEDI GDAS steps for ATMS (activate atms from obs list and manually staged atms bias files in gdas.20210731/18/analysis/atmos)

1. gdasprep --- successful
2. gdasprepatm --- successful
3. gdasatmanlinit --- successful
4. gdasatmanlrun (npe_atmanlrun_gfs=384) --- failed with the error messages: stripack_addnod: colinear points and Command exited with status 137 There are still problems in running the analysis step. The problem seems to be related to memory.

This PR fixed the inconsistency in the feature/gdas_validation with JEDI and G-W. But the JEDI experiment failed at gdasatmanlrun step.

@RussTreadon-NOAA Your test in the gdasatmanlrun also failed, correct?

[RussTreadon-NOAA]

@emilyhcliu , did you run gdasatmanl with CASE_ANL=C768? This no longer works. Try CASE_ANL=C384. @CoryMartin-NOAA recommends opening a fv3-jedi issue to report this issue.

[emilyhcliu]

@emilyhcliu , did you run gdasatmanl with CASE_ANL=C768? This no longer works. Try CASE_ANL=C384. @CoryMartin-NOAA recommends opening a fv3-jedi issue to report this issue. @RussTreadon-NOAA OK. Thanks. Let me try the lower resolution with c384.

[emilyhcliu]

@RussTreadon-NOAA The low-resolution run was completed successfully for ATMS.

This PR is approved!!

[RussTreadon-NOAA]

Thank you @emilyhcliu for the approval