Updates are mainly to the SkewT driver script. I set up the script so that it can run as a single job simultaneously with the UPP metatask. Inside the script, it does this:
Checks to make sure that the 3D native level grib files are “old enough” (3 mins)
If a file is ready, it creates all the SkewTs from that forecast hour in parallel using a subprocess Pool
When the Pool is done it zips the files, so a set for a forecast hour is available for the web just a few minutes after UPP is done. (NCL waits until all are done before zipping, I think.)
It moves on to the next forecast hour. If none are available it waits in increments of a minute until there are files that are ready.
If nothing meaningful has happened for 10 minutes (my arbitrary choice) it releases the compute node. I chose this in case we get held up on some post jobs for a bit.
I ran a test in real-time (as me). The job ran on sjet completed the zipped files for the 18 hr forecast at 16 UTC about 8 minutes after UPP finished. The job held the single node for 72 minutes.
Updates are mainly to the SkewT driver script. I set up the script so that it can run as a single job simultaneously with the UPP metatask. Inside the script, it does this:
I ran a test in real-time (as me). The job ran on sjet completed the zipped files for the 18 hr forecast at 16 UTC about 8 minutes after UPP finished. The job held the single node for 72 minutes.