NOAA-EMC / global-workflow

Global Superstructure/Workflow supporting the Global Forecast System (GFS)

https://global-workflow.readthedocs.io/en/latest

GNU Lesser General Public License v3.0

70 stars 162 forks source link

Eliminate post groups #2667

Closed WalterKolczynski-NOAA closed 2 weeks ago

WalterKolczynski-NOAA commented 4 weeks ago

Description

Eliminates the post groups used for upp and products jobs so that each task only processes one forecast hour. This is more efficient and greatly simplifies downstream dependencies that depend on a specific forecast hour.

Resolves #2666 Refs #2642

Type of change

Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

Is this a breaking change (a change in existing functionality)? NO
Does this change require a documentation update? NO

How has this been tested?

Cycled test on Hercules

Checklist

[x] Any dependent changes have been merged and published
[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] My changes generate no new warnings
[x] New and existing tests pass with my changes
[x] I have made corresponding changes to the documentation if necessary

emcbot commented 2 weeks ago

Experiment C48_ATM FAILED on Hera in /scratch1/NCEPDEV/global/CI/2667/RUNTESTS/C48_ATM_10bcdcba

emcbot commented 2 weeks ago

Experiment C96_atmaerosnowDA FAILED on Hera in /scratch1/NCEPDEV/global/CI/2667/RUNTESTS/C96_atmaerosnowDA_10bcdcba

emcbot commented 2 weeks ago

Experiment C96_atm3DVar FAILED on Hera in /scratch1/NCEPDEV/global/CI/2667/RUNTESTS/C96_atm3DVar_10bcdcba

emcbot commented 2 weeks ago

Experiment C48mx500_3DVarAOWCDA FAILED on Hera in /scratch1/NCEPDEV/global/CI/2667/RUNTESTS/C48mx500_3DVarAOWCDA_10bcdcba

emcbot commented 2 weeks ago

Experiment C96C48_hybatmDA FAILED on Hera in /scratch1/NCEPDEV/global/CI/2667/RUNTESTS/C96C48_hybatmDA_10bcdcba

emcbot commented 2 weeks ago

Experiment C48_S2SWA_gefs FAILED on Hera in /scratch1/NCEPDEV/global/CI/2667/RUNTESTS/C48_S2SWA_gefs_10bcdcba

emcbot commented 2 weeks ago

Experiment C96C48_hybatmDA FAILED on Hera with error logs:

/scratch1/NCEPDEV/global/CI/2667/RUNTESTS/COMROOT/C96C48_hybatmDA_a1ad6f4a/logs/2021122100/enkfgdasearc01.log

Follow link here to view the contents of the above file(s): (link)

emcbot commented 2 weeks ago

Experiment C96C48_hybatmDA FAILED on Hera in /scratch1/NCEPDEV/global/CI/2667/RUNTESTS/C96C48_hybatmDA_a1ad6f4a

aerorahul commented 2 weeks ago

Experiment C96C48_hybatmDA FAILED on Hera with error logs:
/scratch1/NCEPDEV/global/CI/2667/RUNTESTS/COMROOT/C96C48_hybatmDA_a1ad6f4a/logs/2021122100/enkfgdasearc01.log
Follow link here to view the contents of the above file(s): (link)

Source of this error is here. https://github.com/NOAA-EMC/global-workflow/pull/2668/files#r1644852497

emcbot commented 2 weeks ago

Experiment C48_S2SWA_gefs FAILED on Hera in /scratch1/NCEPDEV/global/CI/2667/RUNTESTS/C48_S2SWA_gefs_a1ad6f4a

emcbot commented 2 weeks ago

Experiment C96_atm3DVar FAILED on Hera in /scratch1/NCEPDEV/global/CI/2667/RUNTESTS/C96_atm3DVar_a1ad6f4a

emcbot commented 2 weeks ago

Experiment C96_atmaerosnowDA FAILED on Hera in /scratch1/NCEPDEV/global/CI/2667/RUNTESTS/C96_atmaerosnowDA_a1ad6f4a

emcbot commented 2 weeks ago

Experiment C48_S2SW FAILED on Hera in /scratch1/NCEPDEV/global/CI/2667/RUNTESTS/C48_S2SW_a1ad6f4a

emcbot commented 2 weeks ago

CI Passed Hercules at
Built and ran in directory /work2/noaa/stmp/CI/HERCULES/2667

emcbot commented 2 weeks ago

Experiment C96C48_hybatmDA FAILED on Hera with error logs:

/scratch1/NCEPDEV/global/CI/2667/RUNTESTS/COMROOT/C96C48_hybatmDA_d5cf7918/logs/2021122100/enkfgdasearc01.log

Follow link here to view the contents of the above file(s): (link)

emcbot commented 2 weeks ago

Experiment C96C48_hybatmDA FAILED on Hera in /scratch1/NCEPDEV/global/CI/2667/RUNTESTS/C96C48_hybatmDA_d5cf7918

emcbot commented 2 weeks ago

Experiment C96_atmaerosnowDA FAILED on Hera in /scratch1/NCEPDEV/global/CI/2667/RUNTESTS/C96_atmaerosnowDA_d5cf7918

emcbot commented 2 weeks ago

Experiment C96_atm3DVar FAILED on Hera in /scratch1/NCEPDEV/global/CI/2667/RUNTESTS/C96_atm3DVar_d5cf7918

emcbot commented 2 weeks ago

Experiment C48_S2SWA_gefs FAILED on Hera in /scratch1/NCEPDEV/global/CI/2667/RUNTESTS/C48_S2SWA_gefs_d5cf7918

emcbot commented 2 weeks ago

Experiment C48_S2SW FAILED on Hera in /scratch1/NCEPDEV/global/CI/2667/RUNTESTS/C48_S2SW_d5cf7918

aerorahul commented 2 weeks ago

Experiment C96C48_hybatmDA FAILED on Hera with error logs:
/scratch1/NCEPDEV/global/CI/2667/RUNTESTS/COMROOT/C96C48_hybatmDA_d5cf7918/logs/2021122100/enkfgdasearc01.log
Follow link here to view the contents of the above file(s): (link)

Ref. https://github.com/NOAA-EMC/global-workflow/pull/2668/files#r1646241550 Following the hotfix in #2697, the COM_ATMOS_<>_TMPL variables are also needed to resolve the jinja2 templates (in addition to the COMIN) We need another hotfix for #2697

For some reason, I am not sure why the CI is not catching these.

emcbot commented 2 weeks ago

CI Passed Hera at
Built and ran in directory /scratch1/NCEPDEV/global/CI/2667

Previous