search

NOAA-EMC / global-workflow

Global Superstructure/Workflow supporting the Global Forecast System (GFS)
https://global-workflow.readthedocs.io/en/latest
GNU Lesser General Public License v3.0
75 stars 168 forks source link

GFSv16.1.2 updates #330

Closed KateFriedman-NOAA closed 3 years ago

KateFriedman-NOAA commented 3 years ago

Two tag updates in this implementation:

  1. EMC_post (see updates listed below)
  2. GSI (adjust global_satinfo.txt to disable cris assimilation) - This tag captures changes implemented as GFS v16.1.1a at 12Z on 28 May 2021. The GFS v16.1.1a ARFC placed CrIS-FSR NPP data in monitor mode due to an instrument anomaly. Tag gfsda.v16.1.1a does not alter current operational GFS DA results.

Two updates in new upp_gfsv16_release.v1.1.4 tag:

  1. Triggering postanl job change for reliability from @lgannoaa
  2. Fix for wafs processing failure from @YaliMao-NOAA

The EMC_post/UPP tag upp_gfsv16_release.v1.1.4 was created for GFS v16 implementation by @WenMeng-NOAA after PR #328. The GSI tag was documented in issue #173.

@lgannoaa tested post change 1 in ecflow on WCOSS. @YaliMao-NOAA tested post change 2.

From June 11th RFC memo:

RFC 8260 - On WCOSS, placed CrIS NPP channels in monitor mode for the GFS, HRRR, HWRF, and RAP models. This was done to minimize possible degraded analyses due to the CrIS NPP data anomaly/outage that began on May 21, 2021. Implemented on May 28 at 1145Z.

From June 18th RFC memo:

RFC 8313 - Upgrade the GFS to v16.1.2, which includes the following two fixes:

  • an updated UPP GTG subroutine to increase
    • wind threshold to 500 m/s,
    • height threshold to 1.0E7 m, and
    • temperature threshold to 3000 K. Also instead of aborting, only send out “out of bound” warning messages when thresholds are exceeded.
  • an updated GFS post processing script so that the GFS post-processing job will not crash and the generation of GFS pgb files will continue even if the WAFS fails.

These changes address recent failures of the GFS WAFS post-processing job which produce turbulence products, which then caused the generation of GFS pgb files and downstream products to come to a halt. To be implemented on June 22 at 1430Z.

KateFriedman-NOAA commented 3 years ago

Created branch release/gfsv16.1.2 for this update.

KateFriedman-NOAA commented 3 years ago

Tagged release/gfsv16.1.2 branch after updating EMC_post tag: EMC-v16.1.2

KateFriedman-NOAA commented 3 years ago

Steven Earle just confirmed this will go into ops as v16.1.2.

KateFriedman-NOAA commented 3 years ago

Expanding this implementation to include new GSI tag (gfsda.v16.1.1a) that includes adjustment to global_satinfo.txt file to disable cris assimilation. This was a hotfix in operations earlier this week (nwprod/gfsv16.1.1a).

Updated Externals.cfg and sorc/checkout.sh with new GSI tag. Recut EMC-v16.1.2 tag.

KateFriedman-NOAA commented 3 years ago

From last Friday's RFC memo:

RFC 8260 - On WCOSS, placed CrIS NPP channels in monitor mode for the GFS, HRRR, HWRF, and RAP models. This was done to minimize possible degraded analyses due to the CrIS NPP data anomaly/outage that began on May 21, 2021. Implemented on May 28 at 1145Z.

KateFriedman-NOAA commented 3 years ago

From June 18th RFC memo:

RFC 8313 - Upgrade the GFS to v16.1.2, which includes the following two fixes:

  • an updated UPP GTG subroutine to increase
    • wind threshold to 500 m/s,
    • height threshold to 1.0E7 m, and
    • temperature threshold to 3000 K. Also instead of aborting, only send out “out of bound” warning messages when thresholds are exceeded.
  • an updated GFS post processing script so that the GFS post-processing job will not crash and the generation of GFS pgb files will continue even if the WAFS fails.

These changes address recent failures of the GFS WAFS post-processing job which produce turbulence products, which then caused the generation of GFS pgb files and downstream products to come to a halt. To be implemented on June 22 at 1430Z.

KateFriedman-NOAA commented 3 years ago

From early AM SDM log today:

2v. 1430Z - SPA Simon began RFC 8313 - WCOSS gfs.v16.1.2 update. 1747Z - Complete.