search

NOAA-EMC / global-workflow

Global Superstructure/Workflow supporting the Global Forecast System (GFS)
https://global-workflow.readthedocs.io/en/latest
GNU Lesser General Public License v3.0
75 stars 168 forks source link

[NCO Bug] Add restart capability for GFS and GDAS gempak jobs #1250

Closed KateFriedman-NOAA closed 4 months ago

KateFriedman-NOAA commented 1 year ago

Bugzilla #1367

Details from NCO in bugzilla:

Please add the restart capability for GFS and GDAS gempak jobs, referred to the "NCEP Central Operations WCOSS 
Implementation Standards, version 11.0.0", page 13, 
https://www.nco.ncep.noaa.gov/idsb/implementation_standards/ImplementationStandards.v11.0.0.pdf?

During the catchup process after system issue, gfs or gdas *gempak jobs could fail on waiting upstream files because 
it ran faster than rerunning upstream jobs. Each time the gemapk jobs will processed from the very beginning and 
alert from 000 hour. Eventually gempak jobs alerted almost two or more times for each hour, which caused large 
queue on dbnet. 

For the restart capability, suggest to add checking if gempak fhr is complete then skip logic in the gempak jobs to 
avoid multiple alerts in rerun. Also best use of the system resource and delivery gempak product efficiently.
aerorahul commented 8 months ago

@HuiyaChuang-NOAA Can you please identify a POC for this NCO bugzilla issue?

GwenChen-NOAA commented 8 months ago

@HuiyaChuang-NOAA Can you please identify a POC for this NCO bugzilla issue?

@aerorahul, I believe that would be me. We will need to fix bug issues #2151, #2152, #2157, #2158, and #2248 first and get the gempak jobs running in global-workflow.