search

NOAA-EMC / EMC_verif-global

Global Forecast System (GFS) verification package using MET and METplus
8 stars 14 forks source link

Update WCOSS2 global-workflow support #123

Closed DavidHuber-NOAA closed 6 months ago

DavidHuber-NOAA commented 6 months ago

When testing https://github.com/NOAA-EMC/global-workflow/pull/2239 on WCOSS2, it appears that a few paths may need to be updated. First in ush/run_verif_global_in_global_workflow.sh:

https://github.com/NOAA-EMC/EMC_verif-global/blob/babaa3638f9b181c399284fe443b90418baf76f2/ush/run_verif_global_in_global_workflow.sh#L230-L234

The HOMEMETplus variable needs to be set.

Once this was set, when running the g2o1 job, errors were reported that a prepbufr directory could not be created. This was proceeded by a number of errors that observation files could not be found, e.g.:

+ job1[48]: /apps/ops/para/libs/intel/19.1.3.304/metplus/3.1.1/ush/master_metplus.py -c /lfs/h2/emc/nems/noscrub/david.huber/GW/gw_160/sorc/verif-global.fd/parm/metplus_config/machine.conf -c /lfs/h2/emc/nems/noscru
b/david.huber/GW/gw_160/sorc/verif-global.fd/parm/metplus_config/metplus_use_cases/METplusV3.1/grid2obs/gather/VSDB.conf
02/16 14:28:40.274 metplus.PB2NC (command_builder.py:122) ERROR: (command_builder.py:539) Could not find OBS_INPUT file /lfs/h2/emc/stmp/david.huber/RUNDIRS/160/metpg2o1.50903/grid2obs_step1/data/prepbufr/prepbufr.n
am.2021110900 using template prepbufr.nam.{valid?fmt=%Y%m%d%H}

A sample log file can be found here: /lfs/h2/emc/ptmp/david.huber/com/160/logs/2021110900/gfsmetpg2o1.log.

Lastly, the pcp1 job is producing errors that the ccpa files cannot be found, e.g.

02/16 14:28:31.174 metplus.PCPCombine (command_runner.py:89) INFO: COMMAND: /apps/ops/para/libs/intel/19.1.3.304/met/9.1.3/bin/pcp_combine -v 2 -sum 20211108_000000 06 20211109_120000 24 /lfs/h2/emc/stmp/david.huber/RUNDIRS/160/metppcp1.125803/precip_step1/metplus_output/make_met_data_by_VALID/pcp_combine/ccpa_accum24hr/160/f036.2021110800.A024hr -pcpdir /lfs/h2/emc/stmp/david.huber/RUNDIRS/160/metppcp1.125803/precip_step1/dat
a/160 -pcprx f.*\.2021110800
02/16 14:28:33.805 metplus.GridStat (compare_gridded_wrapper.py:176) INFO: Processing forecast lead 1 day 12 hours
02/16 14:28:33.962 metplus.GridStat (command_builder.py:122) ERROR: (command_builder.py:539) Could not find OBS_INPUT file /lfs/h2/emc/stmp/david.huber/RUNDIRS/160/metppcp1.125803/precip_step1/data/ccpa_accum24hr/cc
pa.2021110912.24h using template ccpa.{valid?fmt=%Y%m%d%H}.24h
02/16 14:28:33.962 metplus.GridStat (command_builder.py:122) ERROR: (command_builder.py:539) Could not find OBS_INPUT file /lfs/h2/emc/stmp/david.huber/RUNDIRS/160/metppcp1.125803/precip_step1/data/ccpa_accum24hr/cc
pa.2021110912.24h using template ccpa.{valid?fmt=%Y%m%d%H}.24h
02/16 14:28:33.962 metplus.GridStat (command_builder.py:122) ERROR: (command_builder.py:412) Could not find observation file 
02/16 14:28:33Z metppcp1-metplus.GridStat: ERROR:  (command_builder.py:539) Could not find OBS_INPUT file /lfs/h2/emc/stmp/david.huber/RUNDIRS/160/metppcp1.125803/precip_step1/data/ccpa_accum24hr/ccpa.2021110912.24h
 using template ccpa.{valid?fmt=%Y%m%d%H}.24h

A sample log file can be found here: /lfs/h2/emc/ptmp/david.huber/com/160/logs/2021110900/gfsmetppcp1.log.

malloryprow commented 6 months ago

I cleared off data from 2021 at the start of 2024. We were running into disk space issue in vpppg/noscrub with the testing and support of EVS as it moves to operations. Plus, we try to keep two years of data on disk. Can you run for a more recent date?

For adding HOMEMETplus, do you know what it needs added? Do you want to do the changes or me?

DavidHuber-NOAA commented 6 months ago

@malloryprow Ah, OK, I'll try a newer date.

If HOMEMETplus is the only thing that needs updating, I'm happy to do that. Just for reference, it is export HOMEMETplus=/apps/ops/para/libs/intel/19.1.3.304/metplus/3.1.1.

malloryprow commented 6 months ago

Yup, that looks good to me!