GFS v16 workflow development - Githubissues

NOAA-EMC / global-workflow

Global Superstructure/Workflow supporting the Global Forecast System (GFS)

https://global-workflow.readthedocs.io/en/latest

GNU Lesser General Public License v3.0

75 stars 168 forks source link

GFS v16 workflow development #1

Closed KateFriedman-NOAA closed 3 years ago

KateFriedman-NOAA commented 4 years ago

This issue is being opened to track workflow development for GFS v16.

Branch for work: feature/gfsv16b

Please see prior issue comments and commits:

https://vlab.ncep.noaa.gov/redmine/issues/65358

CatherineThomas-NOAA commented 4 years ago

After seeing a banding feature in the VSDB analysis increment plots, the temperature increments were examined in greater detail, comparing netCDF increment files, netCDF and nemsio model and utility output, and post output. The results were documented and sent to post and key GFSv16 developers.

PGB ANL Temperature.pdf

CatherineThomas-NOAA commented 4 years ago

It was speculated that the ensemble forecast failures in early-mid January in v16rt2 could have been caused by a runaway ensemble spread at the model top. The ensemble spread was examined over the v16rt2 period and it was found that while the ensemble spread is larger than it is in operations, the ensemble spread is not growing out of control in the early-mid January timeframe. Spread in v16rt2.pdf

CatherineThomas-NOAA commented 4 years ago

In response to several ensemble forecast failures, the parameters k_split and n_split were changed in the real time parallel. A low resolution parallel testing these changes was run. Preliminary results show that the fit to conventional observations are very similar.

CatherineThomas-NOAA commented 4 years ago

The evaluation of the low resolution parallel with altered k_split and n_split was completed. The previous preliminary results of neutral impact were confirmed.

KateFriedman-NOAA commented 4 years ago

@RussTreadon-NOAA @yangfanglin I updated the prod_util version on WCOSS-Dell to v1.1.4 to capture the post-P1/P2 removal fix for compath.py. Most important change was to modulefiles/module_base.wcoss_dell_p3.

Committed to relevant files at 9b923ac.

KateFriedman-NOAA commented 4 years ago

Created release/gfsv16.0.0 branch off operations branch. Will merge feature/gfsv16b into it and resolve conflicts on the command line.

Will move release notes into branch too.

KateFriedman-NOAA commented 4 years ago

Release branch completed for hand-off and submitted to NCO Friday 10/9.

Now begin post-hand-off tasks:

test release branch on Hera and Orion at all supported resolution combos (make fixes as needed)
test release branch on WCOSS for non-ops resolutions (make fixes as needed)
post fix_nco_gfsv16 fix files on ftp server
merge release branch (via port2orion) to develop branch
do GitHub release of GFSv16.0.0
make changes/adjustments based on NCO review
pull in updated UPP

KateFriedman-NOAA commented 4 years ago

Post hand-off changes (updated 10/16):

Update WAFS dependencies to wait for all f06 through f36 output to be available. Committed at 4b2d26d.
Update FV3 tag to GFS.v16.0.14 for Hera/Orion build support. Committed at 484550c.

KateFriedman-NOAA commented 3 years ago

Committed corrected npe_eobs values to config.resources in release/gfsv16.0.0 branch at 794e6655.

Post hand-off changes (updated 10/20):

Update WAFS dependencies to wait for all f06 through f36 output to be available. Committed at 4b2d26d.
Update FV3 tag to GFS.v16.0.14 for Hera/Orion build support. Committed at 484550c.
Correct npe_eobs values in config.resources. Committed at 794e6655.

KateFriedman-NOAA commented 3 years ago

Committed updated WAFS tag at c09678f.

Post hand-off changes (updated 10/26):

Update WAFS dependencies to wait for all f06 through f36 output to be available. Committed at 4b2d26d.
Update FV3 tag to GFS.v16.0.14 for Hera/Orion build support. Committed at 484550c.
Correct npe_eobs values in config.resources. Committed at 794e665.
Update WAFS tag to gfs_wafs.v6.0.12. Committed at c09678f.

KateFriedman-NOAA commented 3 years ago

Renamed release notes (.txt -> .md) per discussion with Fanglin. Committed at cb6d74e.

Post hand-off changes (updated 10/26):

Update WAFS dependencies to wait for all f06 through f36 output to be available. Committed at 4b2d26d.
Update FV3 tag to GFS.v16.0.14 for Hera/Orion build support. Committed at 484550c.
Correct npe_eobs values in config.resources. Committed at 794e665.
Update WAFS tag to gfs_wafs.v6.0.12. Committed at c09678f.
Rename release notes for v16.0.0 to make .md README format (.txt -> .md). Committed at cb6d74e.

KateFriedman-NOAA commented 3 years ago

Updated WAFS tag to gfs_wafs.v6.0.13 after testing on WCOSS-Dell. Committed at a3b4638.

Post hand-off changes (updated 10/30):

Update WAFS dependencies to wait for all f06 through f36 output to be available. Committed at 4b2d26d.
Update FV3 tag to GFS.v16.0.14 for Hera/Orion build support. Committed at 484550c.
Correct npe_eobs values in config.resources. Committed at 794e665.
Update WAFS tag to gfs_wafs.v6.0.12. Committed at c09678f.
Rename release notes for v16.0.0 to make .md README format (.txt -> .md). Committed at cb6d74e.
Updated WAFS tag to gfs_wafs.v6.0.13. Committed at a3b4638.
Updated jobs/JGFS_ATMOS_POSTSND and jobs/rocoto/postsnd.sh to remove redundant variables in the two scripts and make them work for both EMC and NCO parallels. Committed at 7059334.

KateFriedman-NOAA commented 3 years ago

Post hand-off changes (updated 11/2):

Update WAFS dependencies to wait for all f06 through f36 output to be available. Committed at 4b2d26d.
Update FV3 tag to GFS.v16.0.14 for Hera/Orion build support. Committed at 484550c.
Correct npe_eobs values in config.resources. Committed at 794e665.
Update WAFS tag to gfs_wafs.v6.0.12. Committed at c09678f.
Rename release notes for v16.0.0 to make .md README format (.txt -> .md). Committed at cb6d74e.
Updated WAFS tag to gfs_wafs.v6.0.13. Committed at a3b4638.
Updated jobs/JGFS_ATMOS_POSTSND and jobs/rocoto/postsnd.sh to remove redundant variables in the two scripts and make them work for both EMC and NCO parallels. Committed at 7059334.
Update WAFS tag to gfs_wafs.v6.0.14 and updated dumpjb version to 5.1.0. Committed at 9fa9ffe9.

KateFriedman-NOAA commented 3 years ago

Tagged new hand-off version: EMC-v16.0.2

KateFriedman-NOAA commented 3 years ago

Tagged new hand-off version: EMC-v16.0.3

KateFriedman-NOAA commented 3 years ago

Retagged EMC-v16.0.3 off release/gfsv16.0.0 this morning after commit at e89045b.

KateFriedman-NOAA commented 3 years ago

Created branch release/gfsv16.0.0_to_ops off of operations branch for merging release/gfsv16.0.0 branch into the operations branch post-GFSv16 implementation on March 22nd 12z. Will use release/gfsv16.0.0_to_ops branch to resolve conflicts before submitting PR of release/gfsv16.0.0_to_ops into operations branch.

KateFriedman-NOAA commented 3 years ago

Performed merge of release/gfsv16.0.0 branch into release/gfsv16.0.0_to_ops branch. Resolved conflicts in following files:

[Kate.Friedman@v71a1 release-gfsv16.0.0-to-ops]$ git branch
  operations
* release/gfsv16.0.0_to_ops
[Kate.Friedman@v71a1 release-gfsv16.0.0-to-ops]$ git merge origin/release/gfsv16.0.0 | grep CONFLICT
warning: inexact rename detection was skipped due to too many files.
warning: you may want to set your merge.renamelimit variable to at least 1579 and retry the command.
CONFLICT (content): Merge conflict in util/modulefiles/gfs_util.wcoss_dell_p3
CONFLICT (content): Merge conflict in util/modulefiles/gfs_util.wcoss_cray
CONFLICT (content): Merge conflict in sorc/checkout.sh
CONFLICT (content): Merge conflict in modulefiles/module_base.wcoss_dell_p3
CONFLICT (modify/delete): modulefiles/module_base.wcoss_cray deleted in origin/release/gfsv16.0.0 and modified in HEAD. Version HEAD of modulefiles/module_base.wcoss_cray left in tree.
CONFLICT (content): Merge conflict in modulefiles/gfs_bufr.wcoss_dell_p3
CONFLICT (modify/delete): modulefiles/gdas_trpsfcmv.wcoss_dell_p3 deleted in origin/release/gfsv16.0.0 and modified in HEAD. Version HEAD of modulefiles/gdas_trpsfcmv.wcoss_dell_p3 left in tree.
CONFLICT (modify/delete): modulefiles/gdas_trpsfcmv.wcoss_cray_userlib deleted in origin/release/gfsv16.0.0 and modified in HEAD. Version HEAD of modulefiles/gdas_trpsfcmv.wcoss_cray_userlib left in tree.
CONFLICT (modify/delete): modulefiles/gdas_trpsfcmv.wcoss_cray deleted in origin/release/gfsv16.0.0 and modified in HEAD. Version HEAD of modulefiles/gdas_trpsfcmv.wcoss_cray left in tree.
CONFLICT (modify/delete): modulefiles/gdas_navybull.wcoss_dell_p3 deleted in origin/release/gfsv16.0.0 and modified in HEAD. Version HEAD of modulefiles/gdas_navybull.wcoss_dell_p3 left in tree.
CONFLICT (modify/delete): modulefiles/gdas_navybull.wcoss_cray deleted in origin/release/gfsv16.0.0 and modified in HEAD. Version HEAD of modulefiles/gdas_navybull.wcoss_cray left in tree.

Committed merge to release/gfsv16.0.0_to_ops at de7b54ea.

KateFriedman-NOAA commented 3 years ago

PR #306 merged into operations branch. Operations branch tagged for NCO-v16.0.9 and released.

KateFriedman-NOAA commented 3 years ago

Now that GFSv16 is in operations I will close this issue. Future updates to the v16 system will be documented in their own issues. Thanks for all your hard work everyone!