search

NOAA-EMC / EMC_verif-global

Global Forecast System (GFS) verification package using MET and METplus
8 stars 14 forks source link

Port to Hercules #121

Closed DavidHuber-NOAA closed 6 months ago

DavidHuber-NOAA commented 7 months ago

This ports EMC_verif-global to Hercules. Though there isn't a stack that has been tested for step 2/plotting jobs, this includes a first stab at support of those jobs. Resolves #120

DavidHuber-NOAA commented 7 months ago

A global workflow experiment was run on Hercules from 2021110818 through 2021111900. Grid2Grid and Precip .stat files were generated for the period, but Grid2Obs failed to generate files. This will need to be investigated.

@malloryprow In the meantime, would you mind taking a look at the G2G and Precip files in /work/noaa/global/dhuber/archive/metplus_data/by_VSDB under the experiment name herc_160? Also, the log files can be found here: /work2/noaa/global/dhuber/para/com/herc_160/logs/202111*/gfsmetp*.log.

malloryprow commented 7 months ago

I'll take a look at this later this week. I need to do some things for EVS.

Does Hercules and Orion have a shared file system? I'm surprised the precip would have ran. I've never put any CCPA files over there. I'm assuming the grid2obs failed due to a similar reasons as I have never put any prepbufr files there. What is the rstprod policy on Hercules? Same as Orion?

DavidHuber-NOAA commented 7 months ago

@malloryprow Yes, you are correct about grid2obs -- missing prepbufr files. Yes, Hercules and Orion have a shared file system and they both allow for rstprod data.

I'm curious about the precip case as well -- I'll check that out a little closer.

malloryprow commented 7 months ago

The precip doesn't run the first cycle since it needs more files to build the 24 hour accumulation. So the job was successful that nothing ran like it was suppose to (but there will be no stat file).

DavidHuber-NOAA commented 7 months ago

It looks like the CCPA data is available in /work/noaa/ovp/mrow/obdata/ccpa_accum24hr.

DavidHuber-NOAA commented 7 months ago

And at least on first glance, the precip stat file look OK in /work/noaa/global/dhuber/archive/metplus_data/by_VSDB/precip/ccpa_accum24hr/00Z/herc_160.

malloryprow commented 7 months ago

Interesting! Hmmm okay I'll look more into this later this week. I'd love to see if we can get the standalone version working too, so that people can make plots.

malloryprow commented 7 months ago

RE: gfsmetpg2o not finding the prepbufr files...

It is looking for the rstprod files at /work/noaa/ovp/mrow/prepbufr/nam which is where the non-restricted files reside. There are a few places in ush/get_data_files.py that need to be updated to include Hercules like on line 1697.

malloryprow commented 7 months ago

Okay I committed those changes directly to the branch. Pull those in and we can see if the next gfsmetpg2o run goes okay.

DavidHuber-NOAA commented 6 months ago

I finally got back to a place where I could run the GW with EMC_verif-global enabled to try these changes out and they seem to have worked! Would you mind checking out the logs (/work2/noaa/global/dhuber/para/com/herc_final_160/logs/*/gfsmetp*) and the .stat products (/work/noaa/global/dhuber/archive/metplus_data/by_VSDB under the experiment name herc_final_160)?

DavidHuber-NOAA commented 6 months ago

@malloryprow Marking this PR ready for review.

malloryprow commented 6 months ago

It all looks good to me! I'll merge the PR, and then the new commit will need to be updated for the g-w, right?

DavidHuber-NOAA commented 6 months ago

Yep, I'll get it into PR https://github.com/NOAA-EMC/global-workflow/pull/2239.