search

NOAA-EMC / GDASApp

Global Data Assimilation System Application
GNU Lesser General Public License v2.1
14 stars 28 forks source link

Update hashes for JEDI submodules #1117

Closed RussTreadon-NOAA closed 1 month ago

RussTreadon-NOAA commented 1 month ago

The hashes for the core JEDI component submodules in sorc/ are several weeks old. This issue is opened to document the updating of the hashes for these JEDI components.

Note: The updating of sorc/soca is documented in GDASApp issue #1093

RussTreadon-NOAA commented 1 month ago

Work for this issue will be done in RussTreadon-NOAA:/feature/submodules

RussTreadon-NOAA commented 1 month ago

Commit d480014 to feature/submodules points oops at bugfix/lgetkf_addincvars at 912da6d. The current head of oops develop, 8b50896, causes test_gdasapp_atm_jjob_ens_run to abort with the error message described in OOPS PR #2645. The changes in bugfix/lgetkf_addincvars are required for this ctest to pass. The oops submodule hash in feature/submodules will be updated once PR #2645 is merged into oops develop

RussTreadon-NOAA commented 1 month ago

The updated JEDI hashes bring in changes to the names of several radiance bias correction variables.

old new
lapse_rate lapseRate
lapse_rate_order_2 lapseRate_order_2
emissivity emissivityJacobian
scan_angle sensorScanAngle
scan_angle_order_2 sensorScanAngle_order_2
scan_angle_order_3 sensorScanAngle_order_3
scan_angle_order_4 sensorScanAngle_order_4

As such, the GDASApp radiance bias correction conversion script needs to be updated accordingly. Required changes are made to

at ba81096 and 18b6058

The var and ens ctests assimilate amsua_n19 data with bias correction. The bias correction variable names encoded in the following bias correction files need to be updated in order for the var and ens analysis ctests to pass.

These files have been updated and are available in the following directory on Orion

Orion-login-3:/work2/noaa/da/rtreadon/CI/GDASApp/data/lowres/gdas.20210323/12/analysis/atmos$ ls -l gdas.t12z.amsua_n19.satbias*
-rw-r--r-- 1 rtreadon da 22260 May 18 13:00 gdas.t12z.amsua_n19.satbias_cov.nc4
-rw-r--r-- 1 rtreadon da 22260 May 18 13:00 gdas.t12z.amsua_n19.satbias.nc4

@CoryMartin-NOAA : We need to update the amsua_n19 satbias files when feature/submodules is merged into develop

RussTreadon-NOAA commented 1 month ago

In addition to updating variable names in the test data amsua_n19 bias correction files and the GDASApp bias correction conversion scripts, we also need to update bias correction variable names in the following jcb-gdas yamls

Note that similar changes need to be made to the yamls for other radiance data in observations/atmosphere/ The changes for amsua_n19 are mentioned in this issue since GDASApp ctests process amsua_n19 data.

Updating the JEDI submodule hashes also requires changes be made to the following jcb-gdas yamls

In both of these yamls saber block name gsi covariance is replaced with gsi static covariance

jcb-gdas issue #6 documents the addition of the above changes to jcb-gdas branch feature/submodules.

guillaumevernieres commented 1 month ago

@RussTreadon-NOAA , soca fixes are in the feature/submodules_soca branch. I was going to issue a pr into your branch ... but you're working on a fork. Any-chance you could merge my branch into yours?

RussTreadon-NOAA commented 1 month ago

@guillaumevernieres , thank you for the soca updates. I merged feature/submodules_soca into feature/submodules at f3cae35.

sorc/soca in feature/submodules dates back to 3/27/2024 (hash 18f7813d). This is the same soca hash as in the authoritative GDASApp develop.

The current head of soca develop is 98e2be9 from last week. Should I update to a more recent soca hash?

RussTreadon-NOAA commented 1 month ago

Update to soca @ 98e2be9. Build of feature/submodules at d01dac0 aborted with

In file included from /work2/noaa/da/rtreadon/git/global-workflow/submodules/sorc/gdas.cd/bundle/gdas-utils/soca/gdas_socahyb\
ridweights.cc(1):
/work2/noaa/da/rtreadon/git/global-workflow/submodules/sorc/gdas.cd/bundle/gdas-utils/soca/gdas_socahybridweights.h(34): erro\
r: class "soca::Geometry" has no member "latlon"
        geom.latlon(lats, lons, halo);
             ^

compilation aborted for /work2/noaa/da/rtreadon/git/global-workflow/submodules/sorc/gdas.cd/bundle/gdas-utils/soca/gdas_socah\
ybridweights.cc (code 2)
make[2]: *** [gdas-utils/soca/CMakeFiles/gdas_socahybridweights.x.dir/gdas_socahybridweights.cc.o] Error 2
make[2]: Leaving directory `/work2/noaa/da/rtreadon/git/global-workflow/submodules/sorc/gdas.cd/build'
make[1]: *** [gdas-utils/soca/CMakeFiles/gdas_socahybridweights.x.dir/all] Error 2

@guillaumevernieres - ignore this comment. My local working copy did not contain your soca fixes. Let me retest and edit this comment after tests are complete.

RussTreadon-NOAA commented 1 month ago

Update to soca @ 98e2be9. Build of feature/submodules at https://github.com/NOAA-EMC/GDASApp/commit/d01dac058bf207334104708ba9a0e64f026200ef successfully completed.

test_gdasapp tests run with the following results

98% tests passed, 1 tests failed out of 46

Label Time Summary:
gdas-utils    =  22.48 sec*proc (9 tests)
script        =  22.48 sec*proc (9 tests)

Total Test time (real) = 1539.73 sec

The following tests FAILED:
        1827 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_POST (Failed)

Look in log file for test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_POST and see the following traceback

+ JGDAS_GLOBAL_OCEAN_ANALYSIS_POST[35]: /work2/noaa/da/rtreadon/git/global-workflow/submodules/sorc/gdas.cd/scripts/exgdas_global_marine_analysis_post.py
2024-05-20 23:54:55,647 - INFO     - root        : ---------------- Copy from RUNDIR to COMOUT
Traceback (most recent call last):
  File "/work2/noaa/da/rtreadon/git/global-workflow/submodules/sorc/gdas.cd/scripts/exgdas_global_marine_analysis_post.py", line 104, in <module>
    os.path.join(com_ice_analysis, f'{cice_rst_date}.cice_model_anl.res.nc')])
  File "/work2/noaa/da/python/opt/core/miniconda3/4.6.14/envs/gdasapp/lib/python3.7/posixpath.py", line 80, in join
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType

Script exgdas_global_marine_analysis_post.py contains

com_ocean_analysis = os.getenv('COM_OCEAN_ANALYSIS')
com_ice_analysis = os.getenv('COM_ICE_ANALYSIS')
com_ice_restart = os.getenv('COM_ICE_RESTART')

but variable COM_ICE_ANALYSIS is not defined in the parent j-job. g-w PR #2584 adds COM_ICE_ANALYSIS to JGDAS_GLOBAL_OCEAN_ANALYSIS_POST.

We need g-w PRs #2584 and #2592 merged into g-w develop to see all test_gdasapp ctests pass when using forked GDASApp branch feature/submodules.

GDASApp develop is ahead of what g-w can actually run.

Attention @guillaumevernieres , @DavidNew-NOAA , @CoryMartin-NOAA , and @danholdaway

guillaumevernieres commented 1 month ago

Ha, sorry. You need to merge develop into your branch @RussTreadon-NOAA . I commented out the post job for now, I'll turn it back on after my g-w pr is merged.

RussTreadon-NOAA commented 1 month ago

Merge GDASApp develop at 39cf2b2 into RussTreadon-NOAA:feature/submodules. Done at 11e8e3b.

Update working copy of DavidNew-NOAA:feature/atmensanlfv3inc to hash 7ce5b3c. This hash is in sync with the current head, 9aad86f, of g-w develop.

feature/submodules is manually cloned into sorc/gdas.cd in the working copy of David's branch. Build GDASApp inside g-w and rerun test_gdasapp. All soca tests now pass. Unfortunately, three different tests now fail. 

Test project /work2/noaa/da/rtreadon/git/global-workflow/submodules/sorc/gdas.cd/build
      Start 1468: test_gdasapp_util_coding_norms
 1/45 Test #1468: test_gdasapp_util_coding_norms ........................   Passed    4.41 sec
      Start 1469: test_gdasapp_util_ioda_example
 2/45 Test #1469: test_gdasapp_util_ioda_example ........................   Passed    5.59 sec
      Start 1470: test_gdasapp_util_prepdata
 3/45 Test #1470: test_gdasapp_util_prepdata ............................   Passed    2.54 sec
      Start 1471: test_gdasapp_util_rads2ioda
 4/45 Test #1471: test_gdasapp_util_rads2ioda ...........................   Passed    0.87 sec
      Start 1472: test_gdasapp_util_ghrsst2ioda
 5/45 Test #1472: test_gdasapp_util_ghrsst2ioda .........................   Passed    0.15 sec
      Start 1473: test_gdasapp_util_smap2ioda
 6/45 Test #1473: test_gdasapp_util_smap2ioda ...........................   Passed    0.12 sec
      Start 1474: test_gdasapp_util_smos2ioda
 7/45 Test #1474: test_gdasapp_util_smos2ioda ...........................   Passed    0.38 sec
      Start 1475: test_gdasapp_util_viirsaod2ioda
 8/45 Test #1475: test_gdasapp_util_viirsaod2ioda .......................   Passed    0.14 sec
      Start 1476: test_gdasapp_util_icecamsr2ioda
 9/45 Test #1476: test_gdasapp_util_icecamsr2ioda .......................   Passed    0.12 sec
      Start 1811: test_gdasapp_check_python_norms
10/45 Test #1811: test_gdasapp_check_python_norms .......................   Passed    2.54 sec
      Start 1812: test_gdasapp_check_yaml_keys
11/45 Test #1812: test_gdasapp_check_yaml_keys ..........................   Passed    1.42 sec
      Start 1813: test_gdasapp_jedi_increment_to_fv3
12/45 Test #1813: test_gdasapp_jedi_increment_to_fv3 ....................   Passed   11.86 sec
      Start 1814: test_gdasapp_setup_cycled_exp
13/45 Test #1814: test_gdasapp_setup_cycled_exp .........................   Passed    1.75 sec
      Start 1815: test_gdasapp_fv3jedi_fv3inc
14/45 Test #1815: test_gdasapp_fv3jedi_fv3inc ...........................***Failed   27.33 sec
      Start 1816: test_gdasapp_soca_nsst_increment_to_mom6
15/45 Test #1816: test_gdasapp_soca_nsst_increment_to_mom6 ..............   Passed   29.80 sec
      Start 1817: test_gdasapp_soca_prep
16/45 Test #1817: test_gdasapp_soca_prep ................................   Passed    4.45 sec
      Start 1818: test_gdasapp_soca_run_clean
17/45 Test #1818: test_gdasapp_soca_run_clean ...........................   Passed    0.02 sec
      Start 1819: test_gdasapp_soca_setup_obsprep
18/45 Test #1819: test_gdasapp_soca_setup_obsprep .......................   Passed   10.16 sec
      Start 1820: test_gdasapp_soca_JGLOBAL_PREP_OCEAN_OBS
19/45 Test #1820: test_gdasapp_soca_JGLOBAL_PREP_OCEAN_OBS ..............   Passed   42.75 sec
      Start 1821: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_PREP
20/45 Test #1821: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_PREP ....   Passed   42.18 sec
      Start 1822: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT
21/45 Test #1822: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT ....   Passed  138.21 sec
      Start 1823: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_RUN
22/45 Test #1823: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_RUN .....   Passed   42.20 sec
      Start 1824: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_ECEN
23/45 Test #1824: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_ECEN ....   Passed   42.18 sec
      Start 1825: test_gdasapp_soca_copy_scratch
24/45 Test #1825: test_gdasapp_soca_copy_scratch ........................   Passed    1.19 sec
      Start 1826: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT
25/45 Test #1826: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT ...   Passed   42.18 sec
      Start 1827: test_gdasapp_soca_socahybridweights
26/45 Test #1827: test_gdasapp_soca_socahybridweights ...................   Passed   10.15 sec
      Start 1828: test_gdasapp_soca_incr_handler
27/45 Test #1828: test_gdasapp_soca_incr_handler ........................   Passed   10.14 sec
      Start 1829: test_gdasapp_soca_ens_handler
28/45 Test #1829: test_gdasapp_soca_ens_handler .........................   Passed   10.16 sec
      Start 1830: test_gdasapp_snow_create_ens
29/45 Test #1830: test_gdasapp_snow_create_ens ..........................   Passed    0.77 sec
      Start 1831: test_gdasapp_snow_imsproc
30/45 Test #1831: test_gdasapp_snow_imsproc .............................   Passed    3.72 sec
      Start 1832: test_gdasapp_snow_apply_jediincr
31/45 Test #1832: test_gdasapp_snow_apply_jediincr ......................***Failed    1.46 sec
      Start 1833: test_gdasapp_snow_letkfoi_snowda
32/45 Test #1833: test_gdasapp_snow_letkfoi_snowda ......................***Failed    1.57 sec
      Start 1834: test_gdasapp_convert_bufr_adpsfc_snow
33/45 Test #1834: test_gdasapp_convert_bufr_adpsfc_snow .................   Passed    3.02 sec
      Start 1838: test_gdasapp_convert_bufr_adpsfc
34/45 Test #1838: test_gdasapp_convert_bufr_adpsfc ......................   Passed    4.27 sec
      Start 1839: test_gdasapp_convert_gsi_satbias
35/45 Test #1839: test_gdasapp_convert_gsi_satbias ......................   Passed    4.72 sec
      Start 1840: test_gdasapp_setup_atm_cycled_exp
36/45 Test #1840: test_gdasapp_setup_atm_cycled_exp .....................   Passed    0.57 sec
      Start 1841: test_gdasapp_atm_jjob_var_init
37/45 Test #1841: test_gdasapp_atm_jjob_var_init ........................   Passed   44.14 sec
      Start 1842: test_gdasapp_atm_jjob_var_run
38/45 Test #1842: test_gdasapp_atm_jjob_var_run .........................   Passed  106.13 sec
      Start 1843: test_gdasapp_atm_jjob_var_inc
39/45 Test #1843: test_gdasapp_atm_jjob_var_inc .........................   Passed   42.13 sec
      Start 1844: test_gdasapp_atm_jjob_var_final
40/45 Test #1844: test_gdasapp_atm_jjob_var_final .......................   Passed   42.28 sec
      Start 1845: test_gdasapp_atm_jjob_ens_init
41/45 Test #1845: test_gdasapp_atm_jjob_ens_init ........................   Passed   43.71 sec
      Start 1846: test_gdasapp_atm_jjob_ens_run
42/45 Test #1846: test_gdasapp_atm_jjob_ens_run .........................   Passed  458.30 sec
      Start 1847: test_gdasapp_atm_jjob_ens_inc
43/45 Test #1847: test_gdasapp_atm_jjob_ens_inc .........................   Passed  106.13 sec
      Start 1848: test_gdasapp_atm_jjob_ens_final
44/45 Test #1848: test_gdasapp_atm_jjob_ens_final .......................   Passed   42.13 sec
      Start 1849: test_gdasapp_aero_gen_3dvar_yaml
45/45 Test #1849: test_gdasapp_aero_gen_3dvar_yaml ......................   Passed    1.50 sec

93% tests passed, 3 tests failed out of 45

Label Time Summary:
gdas-utils    =  14.33 sec*proc (9 tests)
script        =  14.33 sec*proc (9 tests)

Total Test time (real) = 1394.66 sec

The following tests FAILED:
        1815 - test_gdasapp_fv3jedi_fv3inc (Failed)
        1832 - test_gdasapp_snow_apply_jediincr (Failed)
        1833 - test_gdasapp_snow_letkfoi_snowda (Failed)
Errors while running CTest

Interestingly, rerunning the three failed tests after the above failure results in all three tests passing. 

(gdasapp) Orion-login-2:/work2/noaa/da/rtreadon/git/global-workflow/submodules/sorc/gdas.cd/build$ ctest -R test_gdasapp_fv3jedi_fv3inc
Test project /work2/noaa/da/rtreadon/git/global-workflow/submodules/sorc/gdas.cd/build
    Start 1815: test_gdasapp_fv3jedi_fv3inc
1/1 Test #1815: test_gdasapp_fv3jedi_fv3inc ......   Passed    4.23 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) =   4.75 sec
(gdasapp) Orion-login-2:/work2/noaa/da/rtreadon/git/global-workflow/submodules/sorc/gdas.cd/build$ ctest -R test_gdasapp_snow_apply_jediincr
Test project /work2/noaa/da/rtreadon/git/global-workflow/submodules/sorc/gdas.cd/build
    Start 1832: test_gdasapp_snow_apply_jediincr
1/1 Test #1832: test_gdasapp_snow_apply_jediincr ...   Passed    2.22 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) =   2.68 sec
(gdasapp) Orion-login-2:/work2/noaa/da/rtreadon/git/global-workflow/submodules/sorc/gdas.cd/build$ ctest -R test_gdasapp_snow_letkfoi_snowda
Test project /work2/noaa/da/rtreadon/git/global-workflow/submodules/sorc/gdas.cd/build
    Start 1833: test_gdasapp_snow_letkfoi_snowda
1/1 Test #1833: test_gdasapp_snow_letkfoi_snowda ...   Passed    3.42 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) =   3.88 sec

This behavior needs to be examined and better understood.

RussTreadon-NOAA commented 1 month ago

Hera test Install RussTreadon-NOAA:feature/submodules at b30d103. Run test_gdasapp. Two tests failed

        1856 - test_gdasapp_atm_jjob_var_run (Failed)
        1860 - test_gdasapp_atm_jjob_ens_run (Failed)

due to the reference check after each application finished. This failure is not unexpected. feature/submodules updates the hash for several JEDI repositories. @DavidNew-NOAA explained how to update the references files. Did so and reran with all tests passing

Test project /scratch1/NCEPDEV/da/Russ.Treadon/git/global-workflow/update_gdasapp/sorc/gdas.cd/build
      Start 1483: test_gdasapp_util_coding_norms
 1/47 Test #1483: test_gdasapp_util_coding_norms ........................   Passed    2.93 sec
      Start 1484: test_gdasapp_util_ioda_example
 2/47 Test #1484: test_gdasapp_util_ioda_example ........................   Passed    0.34 sec
      Start 1485: test_gdasapp_util_prepdata
 3/47 Test #1485: test_gdasapp_util_prepdata ............................   Passed    1.03 sec
      Start 1486: test_gdasapp_util_rads2ioda
 4/47 Test #1486: test_gdasapp_util_rads2ioda ...........................   Passed    0.25 sec
      Start 1487: test_gdasapp_util_ghrsst2ioda
 5/47 Test #1487: test_gdasapp_util_ghrsst2ioda .........................   Passed    0.14 sec
      Start 1488: test_gdasapp_util_smap2ioda
 6/47 Test #1488: test_gdasapp_util_smap2ioda ...........................   Passed    0.16 sec
      Start 1489: test_gdasapp_util_smos2ioda
 7/47 Test #1489: test_gdasapp_util_smos2ioda ...........................   Passed    0.15 sec
      Start 1490: test_gdasapp_util_viirsaod2ioda
 8/47 Test #1490: test_gdasapp_util_viirsaod2ioda .......................   Passed    0.14 sec
      Start 1491: test_gdasapp_util_icecamsr2ioda
 9/47 Test #1491: test_gdasapp_util_icecamsr2ioda .......................   Passed    0.14 sec
      Start 1826: test_gdasapp_check_python_norms
10/47 Test #1826: test_gdasapp_check_python_norms .......................   Passed    3.15 sec
      Start 1827: test_gdasapp_check_yaml_keys
11/47 Test #1827: test_gdasapp_check_yaml_keys ..........................   Passed    0.28 sec
      Start 1828: test_gdasapp_jedi_increment_to_fv3
12/47 Test #1828: test_gdasapp_jedi_increment_to_fv3 ....................   Passed    0.83 sec
      Start 1829: test_gdasapp_setup_cycled_exp
13/47 Test #1829: test_gdasapp_setup_cycled_exp .........................   Passed    0.86 sec
      Start 1830: test_gdasapp_fv3jedi_fv3inc
14/47 Test #1830: test_gdasapp_fv3jedi_fv3inc ...........................   Passed   14.12 sec
      Start 1831: test_gdasapp_soca_nsst_increment_to_mom6
15/47 Test #1831: test_gdasapp_soca_nsst_increment_to_mom6 ..............   Passed    3.43 sec
      Start 1832: test_gdasapp_soca_prep
16/47 Test #1832: test_gdasapp_soca_prep ................................   Passed    3.80 sec
      Start 1833: test_gdasapp_soca_run_clean
17/47 Test #1833: test_gdasapp_soca_run_clean ...........................   Passed    0.58 sec
      Start 1834: test_gdasapp_soca_setup_obsprep
18/47 Test #1834: test_gdasapp_soca_setup_obsprep .......................   Passed   23.57 sec
      Start 1835: test_gdasapp_soca_JGLOBAL_PREP_OCEAN_OBS
19/47 Test #1835: test_gdasapp_soca_JGLOBAL_PREP_OCEAN_OBS ..............   Passed   75.62 sec
      Start 1836: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_PREP
20/47 Test #1836: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_PREP ....   Passed   42.47 sec
      Start 1837: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT
21/47 Test #1837: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT ....   Passed   74.53 sec
      Start 1838: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_RUN
22/47 Test #1838: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_RUN .....   Passed   42.49 sec
      Start 1839: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_ECEN
23/47 Test #1839: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_ECEN ....   Passed   74.61 sec
      Start 1840: test_gdasapp_soca_copy_scratch
24/47 Test #1840: test_gdasapp_soca_copy_scratch ........................   Passed    4.46 sec
      Start 1841: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT
25/47 Test #1841: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT ...   Passed   42.63 sec
      Start 1842: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_POST
26/47 Test #1842: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_POST ....   Passed   42.51 sec
      Start 1843: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_VRFY
27/47 Test #1843: test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_VRFY ....   Passed  234.77 sec
      Start 1844: test_gdasapp_soca_socahybridweights
28/47 Test #1844: test_gdasapp_soca_socahybridweights ...................   Passed   10.47 sec
      Start 1845: test_gdasapp_soca_incr_handler
29/47 Test #1845: test_gdasapp_soca_incr_handler ........................   Passed   10.38 sec
      Start 1846: test_gdasapp_soca_ens_handler
30/47 Test #1846: test_gdasapp_soca_ens_handler .........................   Passed   10.44 sec
      Start 1847: test_gdasapp_snow_create_ens
31/47 Test #1847: test_gdasapp_snow_create_ens ..........................   Passed    2.39 sec
      Start 1848: test_gdasapp_snow_imsproc
32/47 Test #1848: test_gdasapp_snow_imsproc .............................   Passed    4.94 sec
      Start 1849: test_gdasapp_snow_apply_jediincr
33/47 Test #1849: test_gdasapp_snow_apply_jediincr ......................   Passed    6.74 sec
      Start 1850: test_gdasapp_snow_letkfoi_snowda
34/47 Test #1850: test_gdasapp_snow_letkfoi_snowda ......................   Passed   12.36 sec
      Start 1851: test_gdasapp_convert_bufr_adpsfc_snow
35/47 Test #1851: test_gdasapp_convert_bufr_adpsfc_snow .................   Passed    4.31 sec
      Start 1852: test_gdasapp_convert_bufr_adpsfc
36/47 Test #1852: test_gdasapp_convert_bufr_adpsfc ......................   Passed    4.18 sec
      Start 1853: test_gdasapp_convert_gsi_satbias
37/47 Test #1853: test_gdasapp_convert_gsi_satbias ......................   Passed    5.82 sec
      Start 1854: test_gdasapp_setup_atm_cycled_exp
38/47 Test #1854: test_gdasapp_setup_atm_cycled_exp .....................   Passed    1.35 sec
      Start 1855: test_gdasapp_atm_jjob_var_init
39/47 Test #1855: test_gdasapp_atm_jjob_var_init ........................   Passed   47.79 sec
      Start 1856: test_gdasapp_atm_jjob_var_run
40/47 Test #1856: test_gdasapp_atm_jjob_var_run .........................   Passed  106.44 sec
      Start 1857: test_gdasapp_atm_jjob_var_inc
41/47 Test #1857: test_gdasapp_atm_jjob_var_inc .........................   Passed   74.35 sec
      Start 1858: test_gdasapp_atm_jjob_var_final
42/47 Test #1858: test_gdasapp_atm_jjob_var_final .......................   Passed   42.43 sec
      Start 1859: test_gdasapp_atm_jjob_ens_init
43/47 Test #1859: test_gdasapp_atm_jjob_ens_init ........................   Passed   47.56 sec
      Start 1860: test_gdasapp_atm_jjob_ens_run
44/47 Test #1860: test_gdasapp_atm_jjob_ens_run .........................   Passed  458.34 sec
      Start 1861: test_gdasapp_atm_jjob_ens_inc
45/47 Test #1861: test_gdasapp_atm_jjob_ens_inc .........................   Passed  106.27 sec
      Start 1862: test_gdasapp_atm_jjob_ens_final
46/47 Test #1862: test_gdasapp_atm_jjob_ens_final .......................   Passed   42.25 sec
      Start 1863: test_gdasapp_aero_gen_3dvar_yaml
47/47 Test #1863: test_gdasapp_aero_gen_3dvar_yaml ......................   Passed    0.81 sec

100% tests passed, 0 tests failed out of 47

Label Time Summary:
gdas-utils    =   5.29 sec*proc (9 tests)
script        =   5.29 sec*proc (9 tests)

Total Test time (real) = 1691.89 sec
RussTreadon-NOAA commented 1 month ago

REMINDER

The names used for certain radiance bias correction terms change with updated JEDI hashes in feature/submodules. As such, the amsua_n19 bias correction files used in the atm_jjob tests must be updated. The updated bias correction files on Hera are in /scratch1/NCEPDEV/da/Russ.Treadon/CI/GDASApp/data/lowres/gdas.20210323/12/analysis/atmos. Variable GDASAPP_TESTDATA was changed in modulefiles/GDAS/hera.intel.lua prior to running the above ctests.

-setenv("GDASAPP_TESTDATA","/scratch1/NCEPDEV/da/Cory.R.Martin/CI/GDASApp/data")
+setenv("GDASAPP_TESTDATA","/scratch1/NCEPDEV/da/Russ.Treadon/CI/GDASApp/data")

The updated amsua_n19 bias correction files will be copied to /scratch1/NCEPDEV/da/Cory.R.Martin/CI/GDASApp/data when PR #1123 is merged into develop. As such, the above change to hera.intel.lua will not be committed to feature/submodules.

A change to GDASAPP_TESTDATA was committed to modulefiles/GDAS/orion.intel.lua. This change must be reverted before PR #1123 is merged into develop

CoryMartin-NOAA commented 1 month ago

Test data has been copied from rtreadon to cmartin on Orion and Hera