NOAA-EMC / global-workflow

Global Superstructure/Workflow supporting the Global Forecast System (GFS)
https://global-workflow.readthedocs.io/en/latest
GNU Lesser General Public License v3.0
70 stars 161 forks source link

Update atmosphere, ocean, and ice post-proceesing jobs to use COMIN/COMOUT #2705

Open HenryWinterbottom-NOAA opened 1 week ago

HenryWinterbottom-NOAA commented 1 week ago

Description

NCO has requested that each COM variable specify whether it is an input or an output. This completes that process for the global-workflow atmosphere and ocean and ice post-processing tasks.

JJOB Script COMIN/COMOUT
JGLOBAL_ATMOS_PRODUCTS exglobal_atmos_products.sh ${COMIN_ATMOS_MASTER}/${PREFIX}master.grb2${fhr3}
JGLOBAL_ATMOS_PRODUCTS exglobal_atmos_products.sh ${COMIN_ATMOS_MASTER}/${PREFIX}sfluxgrb${fhr3}.grib2
JGLOBAL_ATMOS_PRODUCTS exglobal_atmos_products.sh ${COMOUT_ATMOS_GRIB_0p25}/${PREFIX}pgrbv.0p25.${fhr3}
JGLOBAL_ATMOS_PRODUCTS exglobal_atmos_products.sh ${COMOUT_ATMOS_GRIB_0p25}/${PREFIX}pgrb2.0p25.${fhr3}.idx
JGLOBAL_ATMOS_PRODUCTS exglobal_atmos_products.sh ${COMOUT_ATMOS_GRIB_0p25}/${PREFIX}pgrb2b.0p25.${fhr3}
JGLOBAL_ATMOS_PRODUCTS exglobal_atmos_products.sh ${COMOUT_ATMOS_GRIB_0p25}/${PREFIX}pgrb2b.0p25.${fhr3}.idx
JGLOBAL_ATMOS_PRODUCTS exglobal_atmos_products.sh ${COMOUT_ATMOS_GRIB_0p25}/${PREFIX}pgrb2.0p50.${fhr3}
JGLOBAL_ATMOS_PRODUCTS exglobal_atmos_products.sh ${COMOUT_ATMOS_GRIB_0p25}/${PREFIX}pgrb2.0p50.${fhr3}.idx
JGLOBAL_ATMOS_PRODUCTS exglobal_atmos_products.sh ${COMOUT_ATMOS_GRIB_0p25}/${PREFIX}pgrb2b.0p50.${fhr3}
JGLOBAL_ATMOS_PRODUCTS exglobal_atmos_products.sh ${COMOUT_ATMOS_GRIB_0p25}/${PREFIX}pgrb2b.0p50.${fhr3}.idx
JGLOBAL_ATMOS_PRODUCTS exglobal_atmos_products.sh ${COMOUT_ATMOS_GRIB_0p25}/${PREFIX}pgrb2.1p0.${fhr3}
JGLOBAL_ATMOS_PRODUCTS exglobal_atmos_products.sh ${COMOUT_ATMOS_GRIB_0p25}/${PREFIX}pgrb2.1p0.${fhr3}.idx
JGLOBAL_ATMOS_PRODUCTS exglobal_atmos_products.sh ${COMOUT_ATMOS_GRIB_0p25}/${PREFIX}pgrb2b.1p0.${fhr3}
JGLOBAL_ATMOS_PRODUCTS exglobal_atmos_products.sh ${COMOUT_ATMOS_GRIB_0p25}/${PREFIX}pgrb2b.1p0.${fhr3}.idx
JGLOBAL_ATMOS_PRODUCTS exglobal_atmos_products.sh ${COMOUT_ATMOS_GRIB_0p25}/${PREFIX}wgne.${fhr3}
JGLOBAL_ATMOS_PRODUCTS exglobal_atmos_products.sh ${COMOUT_ATMOS_GRIB_0p50}/${PREFIX}wgne.${fhr3}
JGLOBAL_ATMOS_PRODUCTS exglobal_atmos_products.sh ${COMOUT_ATMOS_GRIB_1p0}/${PREFIX}wgne.${fhr3}
JGLOBAL_OCEANICE_PRODUCTS exglobal_oceanice_products.py {{ COMIN_ICE_HISTORY }}/{{ RUN }}.ice.t{{ HOUR }}z.6hr_avg.f{{ FORECAST_HOUR }}.nc
JGLOBAL_OCEANICE_PRODUCTS exglobal_oceanice_products.py {{ COMOUT_ICE_HISTORY }}/{{ RUN }}.ice.t{{ HOUR }}z.6hr_avg.f{{ FORECAST_HOUR }}.nc
JGLOBAL_OCEANICE_PRODUCTS exglobal_oceanice_products.py {{ COMOUT_ICE_NETCDF }}/{{ RUN }}.ice.t{{ HOUR }}z.native.f{{ FORECAST_HOUR }}.nc
JGLOBAL_OCEANICE_PRODUCTS exglobal_oceanice_products.py {{ COMOUT_ICE_GRIB }}/{{ grid }}/{{ RUN }}.ice.t{{ HOUR }}z.{{ grid }}.f{{ FORECAST_HOUR }}.grib2
JGLOBAL_OCEANICE_PRODUCTS exglobal_oceanice_products.py {{ COMOUT_ICE_GRIB }}/{{ grid }}/{{ RUN }}.ice.t{{ HOUR }}z.{{ grid }}.f{{ FORECAST_HOUR }}.grib2.idx
JGLOBAL_OCEANICE_PRODUCTS exglobal_oceanice_products.py {{ COMOUT_OCEAN_HISTORY }}/{{ RUN }}.ocean.t{{ HOUR }}z.6hr_avg.f{{ FORECAST_HOUR }}.nc
JGLOBAL_OCEANICE_PRODUCTS exglobal_oceanice_products.py {{ COMOUT_ICE_NETCDF }}/{{ RUN }}.ocean.t{{ HOUR }}z.native.f{{ FORECAST_HOUR }}.nc
JGLOBAL_OCEANICE_PRODUCTS exglobal_oceanice_products.py {{ COMOUT_ICE_GRIB }}/{{ grid }}/{{ RUN }}.ocean.t{{ HOUR }}z.{{ grid }}.f{{ FORECAST_HOUR }}.grib2
JGLOBAL_OCEANICE_PRODUCTS exglobal_oceanice_products.py {{ COMOUT_OCEAN_GRIB }}/{{ grid }}/{{ RUN }}.ocean.t{{ HOUR }}z.{{ grid }}.f{{ FORECAST_HOUR }}.grib2.idx

Refs #2451

Type of change

Change characteristics

How has this been tested?

       CYCLE                    TASK                       JOBID               STATE         EXIT STATUS     TRIES      DURATION
================================================================================================================================
202103231200             gfsstage_ic                    62186119           SUCCEEDED                   0         1          25.0
202103231200             gfswaveinit                    62186121           SUCCEEDED                   0         1          34.0
202103231200                 gfsfcst                    62186676           SUCCEEDED                   0         1        2256.0
202103231200    gfsatmos_prod_f000-f006                    62188265           SUCCEEDED                   0         1         453.0
202103231200    gfsatmos_prod_f009-f015                    62188404           SUCCEEDED                   0         1         456.0
202103231200    gfsatmos_prod_f018-f024                    62188403           SUCCEEDED                   0         1         455.0
202103231200    gfsatmos_prod_f027-f033                    62188495           SUCCEEDED                   0         1         409.0
202103231200    gfsatmos_prod_f036-f042                    62188496           SUCCEEDED                   0         1         411.0
202103231200    gfsatmos_prod_f045-f051                    62188851           SUCCEEDED                   0         1         405.0
202103231200    gfsatmos_prod_f054-f060                    62188852           SUCCEEDED                   0         1         403.0
202103231200    gfsatmos_prod_f063-f069                    62189027           SUCCEEDED                   0         1         403.0
202103231200    gfsatmos_prod_f072-f078                    62189028           SUCCEEDED                   0         1         400.0
202103231200    gfsatmos_prod_f081-f087                    62189183           SUCCEEDED                   0         1         410.0
202103231200    gfsatmos_prod_f090-f096                    62189186           SUCCEEDED                   0         1         404.0
202103231200    gfsatmos_prod_f099-f105                    62189321           SUCCEEDED                   0         1         402.0
202103231200    gfsatmos_prod_f108-f114                    62189313           SUCCEEDED                   0         1         407.0
202103231200    gfsatmos_prod_f117-f120                    62189460           SUCCEEDED                   0         1         280.0
202103231200              gfstracker                    62189823           SUCCEEDED                   0         1          70.0
202103231200              gfsgenesis                    62189824           SUCCEEDED                   0         1         533.0
202103231200    gfsocean_prod_f006-f018                    62188415           SUCCEEDED                   0         1         115.0
202103231200    gfsocean_prod_f024-f036                    62188526           SUCCEEDED                   0         1          83.0
202103231200    gfsocean_prod_f042-f054                    62188899           SUCCEEDED                   0         1          80.0
202103231200    gfsocean_prod_f060-f072                    62189046           SUCCEEDED                   0         1          82.0
202103231200    gfsocean_prod_f078-f090                    62189197           SUCCEEDED                   0         1          83.0
202103231200    gfsocean_prod_f096-f108                    62189323           SUCCEEDED                   0         1          84.0
202103231200    gfsocean_prod_f114-f120                    62189485           SUCCEEDED                   0         1          59.0
202103231200    gfsice_prod_f006-f018                    62188416           SUCCEEDED                   0         1         101.0
202103231200    gfsice_prod_f024-f036                    62188527           SUCCEEDED                   0         1          61.0
202103231200    gfsice_prod_f042-f054                    62188900           SUCCEEDED                   0         1          59.0
202103231200    gfsice_prod_f060-f072                    62189047           SUCCEEDED                   0         1          62.0
202103231200    gfsice_prod_f078-f090                    62189198           SUCCEEDED                   0         1          63.0
202103231200    gfsice_prod_f096-f108                    62189324           SUCCEEDED                   0         1          64.0
202103231200    gfsice_prod_f114-f120                    62189486           SUCCEEDED                   0         1          46.0
202103231200          gfswavepostsbs                    62187812           SUCCEEDED                   0         1        2330.0
202103231200          gfswavepostpnt                    62189325           SUCCEEDED                   0         1        7445.0
202103231200                 gfsarch                    62194484           SUCCEEDED                   0         1          46.0
202103231200              gfscleanup    

Change characteristics

Checklist

aerorahul commented 1 week ago

@HenryWinterbottom-NOAA There is not a single COMIN in the table? Where is the data for generating the products coming from?

HenryWinterbottom-NOAA commented 1 week ago

That table was generated prior to the original PR. It will be updated following addressing your comments and the tests pass.

WalterKolczynski-NOAA commented 6 days ago

That table was generated prior to the original PR. It will be updated following addressing your comments and the tests pass.

Creating table should be the first step in these PRs, before you make any changes. Then you use it to determine what should be IN and what is OUT.

aerorahul commented 6 days ago

@HenryWinterbottom-NOAA In the description of the PR, How has this been tested, it says, C48_ATM on Hera, and then the rocotostat shows gfsocean, gfsice and gfswave jobs run as part of that experiment. How is that possible? C48_ATM only runs the atmosphere.

HenryWinterbottom-NOAA commented 6 days ago

@HenryWinterbottom-NOAA In the description of the PR, How has this been tested, it says, C48_ATM on Hera, and then the rocotostat shows gfsocean, gfsice and gfswave jobs run as part of that experiment. How is that possible? C48_ATM only runs the atmosphere.

Typo.