search

NOAA-EMC / EMC_verif-global

Global Forecast System (GFS) verification package using MET and METplus
8 stars 14 forks source link

Run metp jobs on the last cycle of CDATE #131

Closed DavidHuber-NOAA closed 1 month ago

DavidHuber-NOAA commented 1 month ago

This modifies the metp jobs so they only run on the last GFS cycle of a given CDATE, i.e. 00Z when gfs_cyc=1, 12Z when gfs_cyc=2, or 18Z when gfs_cyc=4.

This also adds special cases for experiments that start/stop on the same CDATE and/or stop before the last GFS cycle of a given CDATE. For example, if an experiment with gfs_cyc=2, SDATE_GFS=2024010100, and EDATE_GFS=2024030100, METplus jobs will run on 2024010112, 2024010212, and 2024010300 for cycles [00, 12], [00, 12], and [00], respectively.

Refs noaa-emc/global-workflow#2790

malloryprow commented 1 month ago

Thanks for this fix! I think it is an artifact left over from when VRFY_BACKHRS was defaulted to and usually ran with 24. Let me know when things are good to go.

DavidHuber-NOAA commented 1 month ago

I've tested this with gfs_cyc=4, ending on a 06 cycle. I'm also going to test it with gfs_cyc=12 ending on a 00 cycle. Once complete, I will open this for review.

DavidHuber-NOAA commented 1 month ago

@malloryprow Could you also bring over the precipitation data to WCOSS2 (i.e. the ccpa_accum24hr files in /lfs/h2/emc/vpppg/noscrub/emc.vpppg/verification/global/archive/obs_data/ccpa_accum24hr) for 2021122000 through 2021122500?

malloryprow commented 1 month ago

@DavidHuber-NOAA Done!

DavidHuber-NOAA commented 1 month ago

Thanks!

DavidHuber-NOAA commented 1 month ago

I made an update to set_init_valid_fhr_info.py to calculate forecast intervals to handle special cases (e.g. fcyc_list=['00','06'], fcyc_list=['12','18'], fcyc_list=['12'], etc) and I have tested this solution for all cases that I can find.

malloryprow commented 1 month ago

Great to hear you found a solution! Has the offended gfsmetpcp1 been run with the changes? If so, was the output what was expected?

DavidHuber-NOAA commented 1 month ago

Yes, I'll provide that info in the global-workflow PR.

malloryprow commented 1 month ago

I looked at the lastest log from the g-w run.

python_gen_env_vars.sh[14]: precip1_ccpa_accum24hr_fhr_list='30, 36' python_gen_env_vars.sh[15]: export precip1_ccpa_accum24hr_fhr_beg=30 python_gen_env_vars.sh[15]: precip1_ccpa_accum24hr_fhr_beg=30 python_gen_env_vars.sh[16]: export precip1_ccpa_accum24hr_fhr_end=36 python_gen_env_vars.sh[16]: precip1_ccpa_accum24hr_fhr_end=36

And the end of the log files looks good to with the final stat files being generated and moved to the archive. Thanks for you work on this @DavidHuber-NOAA! I appreciate it!

Merging now!