Make the seaice DA actually do something ...

guillaumevernieres commented 2 months ago

fixes #1087

emcbot commented 2 months ago

Automated Global-Workflow GDASApp Testing Results: Machine: orion

Start: Mon May  6 09:19:50 CDT 2024 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Mon May  6 10:00:31 CDT 2024
---------------------------------------------------
Tests:                                  *Failed*
Tests: Failed at Mon May  6 10:45:15 CDT 2024
Tests: 87% tests passed, 6 tests failed out of 45
    1762 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_POST (Failed)
    1775 - test_gdasapp_atm_jjob_var_run (Failed)
    1776 - test_gdasapp_atm_jjob_var_inc (Failed)
    1777 - test_gdasapp_atm_jjob_var_final (Failed)
    1779 - test_gdasapp_atm_jjob_ens_run (Failed)
    1780 - test_gdasapp_atm_jjob_ens_final (Failed)
Tests: see output at /work2/noaa/stmp/cmartin/CI/GDASApp/workflow/PR/1092/global-workflow/sorc/gdas.cd/build/log.ctest

emcbot commented 2 months ago

Automated Global-Workflow GDASApp Testing Results: Machine: hera

Start: Mon May  6 14:27:07 UTC 2024 on hfe10
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Mon May  6 15:15:44 UTC 2024
---------------------------------------------------
Tests:                                  *Failed*
Tests: Failed at Mon May  6 16:17:03 UTC 2024
Tests: 69% tests passed, 14 tests failed out of 45
    1755 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_PREP (Failed)
    1756 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT (Failed)
    1757 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_RUN (Failed)
    1760 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT (Failed)
    1761 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_POST (Failed)
    1762 - test_gdasapp_soca_socahybridweights (Failed)
    1763 - test_gdasapp_soca_incr_handler (Failed)
    1764 - test_gdasapp_soca_ens_handler (Failed)
    1774 - test_gdasapp_atm_jjob_var_run (Failed)
    1775 - test_gdasapp_atm_jjob_var_inc (Failed)
    1776 - test_gdasapp_atm_jjob_var_final (Failed)
    1778 - test_gdasapp_atm_jjob_ens_run (Failed)
    1779 - test_gdasapp_atm_jjob_ens_final (Failed)
Tests: see output at /scratch1/NCEPDEV/da/Cory.R.Martin/CI/GDASApp/workflow/PR/1092/global-workflow/sorc/gdas.cd/build/log.ctest

guillaumevernieres commented 2 months ago

I don't understand the failures of the marine DA on hera ... Only post should have failed
Failure on orion is related to a change in the g-w that hasn't been merge yet
I suppose we were expecting things to be broken for the atmosphere? Pinging @danholdaway , @CoryMartin-NOAA or @RussTreadon-NOAA ... in no particular order! It's complaining about not finding an executable.

RussTreadon-NOAA commented 2 months ago

@guillaumevernieres : I was looking at log files in /scratch1/NCEPDEV/da/Cory.R.Martin/CI/GDASApp/workflow/PR/1092/global-workflow/sorc/gdas.cd/build/gdas/test/atm/global-workflow/testrun when the directory disappeared.

The atm_jjob tests probably failed because g-w has not yet been updated to be consistent with GDASApp PR #1075.

I just finished building g-w PR #2565 on Hera. All test_gdasapp jobs pass including the atm var and ens jobs.

      Start 1772: test_gdasapp_setup_atm_cycled_exp
37/45 Test #1772: test_gdasapp_setup_atm_cycled_exp .....................   Passed    2.45 sec
      Start 1773: test_gdasapp_atm_jjob_var_init
38/45 Test #1773: test_gdasapp_atm_jjob_var_init ........................   Passed   48.96 sec
      Start 1774: test_gdasapp_atm_jjob_var_run
39/45 Test #1774: test_gdasapp_atm_jjob_var_run .........................   Passed  106.39 sec
      Start 1775: test_gdasapp_atm_jjob_var_inc
40/45 Test #1775: test_gdasapp_atm_jjob_var_inc .........................   Passed  106.61 sec
      Start 1776: test_gdasapp_atm_jjob_var_final
41/45 Test #1776: test_gdasapp_atm_jjob_var_final .......................   Passed   42.83 sec
      Start 1777: test_gdasapp_atm_jjob_ens_init
42/45 Test #1777: test_gdasapp_atm_jjob_ens_init ........................   Passed   79.26 sec
      Start 1778: test_gdasapp_atm_jjob_ens_run
43/45 Test #1778: test_gdasapp_atm_jjob_ens_run .........................   Passed  554.52 sec
      Start 1779: test_gdasapp_atm_jjob_ens_final
44/45 Test #1779: test_gdasapp_atm_jjob_ens_final .......................   Passed   74.43 sec
      Start 1780: test_gdasapp_aero_gen_3dvar_yaml
45/45 Test #1780: test_gdasapp_aero_gen_3dvar_yaml ......................   Passed    1.06 sec

100% tests passed, 0 tests failed out of 45

Label Time Summary:
gdas-utils    =   9.55 sec*proc (9 tests)
script        =   9.55 sec*proc (9 tests)

Total Test time (real) = 1605.60 sec

JEDI ATM CI is now running on Hera.

guillaumevernieres commented 2 months ago

Sorry @RussTreadon-NOAA , I asked @CoryMartin-NOAA to remove the pr directory. The issue in the marine DA didn't make any sense.

emcbot commented 2 months ago

Automated Global-Workflow GDASApp Testing Results: Machine: hera

Start: Mon May  6 21:16:51 UTC 2024 on hfe07
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Mon May  6 21:58:50 UTC 2024
---------------------------------------------------
Tests:                                  *Failed*
Tests: Failed at Mon May  6 22:17:29 UTC 2024
Tests: 89% tests passed, 5 tests failed out of 45
    1774 - test_gdasapp_atm_jjob_var_run (Failed)
    1775 - test_gdasapp_atm_jjob_var_inc (Failed)
    1776 - test_gdasapp_atm_jjob_var_final (Failed)
    1778 - test_gdasapp_atm_jjob_ens_run (Failed)
    1779 - test_gdasapp_atm_jjob_ens_final (Failed)
Tests: see output at /scratch1/NCEPDEV/da/Cory.R.Martin/CI/GDASApp/workflow/PR/1092/global-workflow/sorc/gdas.cd/build/log.ctest

NOAA-EMC / GDASApp

Make the seaice DA actually do something ... #1092