NOAA-EMC / global-workflow

Global Superstructure/Workflow supporting the Global Forecast System (GFS)
https://global-workflow.readthedocs.io/en/latest
GNU Lesser General Public License v3.0
75 stars 168 forks source link

land surface upgrades for hr4 #2787

Closed HelinWei-NOAA closed 3 months ago

HelinWei-NOAA commented 3 months ago

Description

This PR:

-->

Type of change

Change characteristics

How has this been tested?

Checklist

emcbot commented 3 months ago

Experiment C48_ATM FAILED on Hera with error logs:

/scratch1/NCEPDEV/global/CI/2787/RUNTESTS/COMROOT/C48_ATM_54e5d447/logs/2021032312/gfsmetpg2o1.log

Follow link here to view the contents of the above file(s): (link)

emcbot commented 3 months ago

Experiment C48_ATM FAILED on Hera in /scratch1/NCEPDEV/global/CI/2787/RUNTESTS/EXPDIR/C48_ATM_54e5d447

emcbot commented 3 months ago

Experiment C48_S2SW FAILED on Hera with error logs:

/scratch1/NCEPDEV/global/CI/2787/RUNTESTS/COMROOT/C48_S2SW_54e5d447/logs/2021032312/gfsmetpg2o1.log

Follow link here to view the contents of the above file(s): (link)

emcbot commented 3 months ago

Experiment C48_S2SW FAILED on Hera in /scratch1/NCEPDEV/global/CI/2787/RUNTESTS/EXPDIR/C48_S2SW_54e5d447

emcbot commented 3 months ago

CI Failed Hera at
Built and ran in directory /scratch1/NCEPDEV/global/CI/2787



Experiment C48_ATM_54e5d447 Terminated with 0
FAIL
FAIL tasks failed and 1 dead at Thu Aug  1 10:31:18 UTC 2024
Experiment C48_ATM_54e5d447 Terminated: *FAIL*
Error logs:
/scratch1/NCEPDEV/global/CI/2787/RUNTESTS/COMROOT/C48_ATM_54e5d447/logs/2021032312/gfsmetpg2o1.log
Experiment C48mx500_3DVarAOWCDA_54e5d447 Completed 2 Cycles: *SUCCESS* at Thu Aug  1 10:37:23 UTC 2024
Experiment C96_atm3DVar_54e5d447 Completed 3 Cycles: *SUCCESS* at Thu Aug  1 11:38:27 UTC 2024
Experiment C96C48_hybatmDA_54e5d447 Completed 3 Cycles: *SUCCESS* at Thu Aug  1 11:38:28 UTC 2024
Experiment C48_S2SWA_gefs_54e5d447 Completed 1 Cycles: *SUCCESS* at Thu Aug  1 11:42:45 UTC 2024
Experiment C48_S2SW_54e5d447 Terminated with 0
FAIL
FAIL tasks failed and 1 dead at Thu Aug  1 12:16:00 UTC 2024
Experiment C48_S2SW_54e5d447 Terminated: *FAIL*
Error logs:
/scratch1/NCEPDEV/global/CI/2787/RUNTESTS/COMROOT/C48_S2SW_54e5d447/logs/2021032312/gfsmetpg2o1.log
Experiment C96_atmaerosnowDA_54e5d447 Completed 3 Cycles: *SUCCESS* at Thu Aug  1 12:21:08 UTC 2024
DavidHuber-NOAA commented 3 months ago

/scratch1/NCEPDEV/global/CI/2787/RUNTESTS/COMROOT/C48_S2SW_54e5d447/logs/2021032312/gfsmetpg2o1.log

This failure is a bug in METplus. See this comment. One potential workaround would be to decrease the number of tasks for Grid2Obs to 1. However, Grid2Obs already takes a long time to run in operations and decreasing the task count from 4 to 1 would significantly increase that runtime.

Since METplus version 9 is no longer supported by the DTC, the likelihood of a bug fix is small.

For the time being, I will manually set the number of METplus tasks to 1 and continue testing.

DavidHuber-NOAA commented 3 months ago

I am not the owner of the CI tests on Hera, so I cannot relaunch this job after all. However, the gfsmetpg2o1 job is the only failure and I believe that means this is a passed test, so I will set the label as such.