NOAA-GSL / pygraf

ADB's Python graphics package
MIT License
10 stars 11 forks source link

Combine prs and nat grib files. #126

Closed christinaholtNOAA closed 2 years ago

christinaholtNOAA commented 2 years ago

This PR adds options to concatenate grib files when multiple paths and file templates are provided. When combined, the resulting file will also remove all duplicated entries. The script self-cleans those combined files upon exit.

This work also addresses what should happen when a forecast hour is not found. If a grib input file is not found within the waiting period, all accumulated variables will be removed from the subsequent forecast hours, and all other images should be generated.

The feature that allows accumulated variables to be plotted a single forecast lead time (an -all_leads flag with -f 6, for example), should now work as expected. If combining files in this mode, all files must be present. A real-time feature to wait on the file has been disabled since this this is not how real-time cases are run.

Should address Issues #94, #111, #85, and #84.

christinaholtNOAA commented 2 years ago

Great question!

The first instance (first_fcst = 6 if cla.images[0] == 'global' else 1) is using the following logic, and we want to catch 0, 1, and 6 for the relevant cases:

if fcst_hour <= first_fcst:
        filenames['01fcst'].append(filename)

And then the second instance (first_fcst = 6 if cla.images[0] == 'global' else 0) has this logic which should start the range from either 0 or 6 depending on the model:

for fhr in range(first_fcst, int(cla.fcst_hour[0]), fcst_inc):

I think that's right...

christinaholtNOAA commented 2 years ago

@Brian-Jamison That's a great point! I think we will really need to consider how we want to wait and submit these jobs. The code will wait for a while to make sure the real-time files are available, but maybe we want to delay the start of the runs until nat files start showing up.