EdwardSnyder-NOAA commented 6 months ago

DESCRIPTION OF CHANGES:

Adding three additional UFS Case Studies (2019 Hurricane Barry, 2019 Halloween Storm, and 2020 July CAPE) to the workflow end-to-end testing process as required by one of our SI features. The UFS Case studies test yamls were created and added to the gaea-c5, derecho, and hera.gnu.com coverage suite files as well as the comprehensive suite files. Removed Cheyenne related logic from current WE2E UFS Case Study (2020_CAD) and extended fcst wall time for it.

Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[X] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

TESTS CONDUCTED:

[ ] hera.intel
[ ] orion.intel
[ ] hercules.intel
[ ] cheyenne.intel
[ ] cheyenne.gnu
[ ] derecho.intel
[ ] gaea.intel
[ ] gaeac5.intel
[ ] jet.intel
[ ] wcoss2.intel
[X] NOAA Cloud (indicate which platform) ran test on PW AWS
[ ] Jenkins
[X] fundamental test suite ran on PW AWS
[ ] comprehensive tests (specify which if a subset was used)

2020 CAPE Results CAPE results from the FV3_GFS_v16 physics suite compare well with the GFS_v15p2 physic suite used in the case study. sfcape_conus_f024

2019 Halloween Results 10m surface winds match well with the RAP Analysis used in the case study. 10mwind_conus_f084

DEPENDENCIES:

DOCUMENTATION:

ISSUE:

CHECKLIST

[X] My code follows the style guidelines in the Contributor's Guide
[X] I have performed a self-review of my own code using the Code Reviewer's Guide
[X] I have commented my code, particularly in hard-to-understand areas
[ ] My changes need updates to the documentation. I have made corresponding changes to the documentation
[X] My changes do not require updates to the documentation (explain).
[X] My changes generate no new warnings
[X] New and existing tests pass with my changes
[X] Any dependent changes have been merged and published

LABELS (optional):

A Code Manager needs to add the following labels to this PR:

[ ] Work In Progress
[ ] bug
[ ] enhancement
[ ] documentation
[ ] release
[ ] high priority
[ ] run_ci
[ ] run_we2e_fundamental_tests
[ ] run_we2e_comprehensive_tests
[ ] Needs Cheyenne test
[ ] Needs Jet test
[ ] Needs Hera test
[ ] Needs Orion test
[ ] help wanted

CONTRIBUTORS (optional):

RatkoVasic-NOAA commented 6 months ago

new tests passed on Hera:

All 3 experiments finished
Calculating core-hour usage and printing final summary
----------------------------------------------------------------------------------------------------
Experiment name                                                  | Status    | Core hours used
----------------------------------------------------------------------------------------------------
2020_CAPE_20240226235238                                           COMPLETE              33.44
2019_hurricane_barry_20240226235240                                COMPLETE              32.83
2019_halloween_storm_20240226235242                                COMPLETE              32.32
----------------------------------------------------------------------------------------------------
Total                                                              COMPLETE              98.59

Detailed summary written to /scratch2/NCEPDEV/fv3-cam/Ratko.Vasic/1043/expt_dirs/WE2E_summary_20240227021304.txt

MichaelLueken commented 5 months ago

The Jenkins tests on Hera GNU, Hera Intel, Hercules, and Orion have successfully passed.

Two WE2E coverage tests on Jet failed - custom_ESGgrid_Great_Lakes_snow_8km and get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_netcdf_2022060112_48h. These failures are due to get_obs_nohrsc and make_lbcs jobs hanging, respectively. Reruns of these two tests are happening now.

There was a disconnect failure on Derecho, but the tests were resubmitted and all successfully passed.

Once the Jet tests pass, I will move forward with merging this PR.

MichaelLueken commented 5 months ago

The summary for Derecho WE2E coverage test:

----------------------------------------------------------------------------------------------------
Experiment name                                                  | Status    | Core hours used 
----------------------------------------------------------------------------------------------------
custom_ESGgrid_IndianOcean_6km_20240228061703                      COMPLETE              25.95
grid_RRFS_CONUS_13km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16_plot_20  COMPLETE              43.15
grid_RRFS_CONUS_25km_ics_NAM_lbcs_NAM_suite_GFS_v16_2024022806170  COMPLETE              50.00
grid_RRFS_CONUScompact_13km_ics_HRRR_lbcs_RAP_suite_HRRR_20240228  COMPLETE              35.16
grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_RRFS_v1beta_2  COMPLETE              20.82
grid_SUBCONUS_Ind_3km_ics_HRRR_lbcs_HRRR_suite_HRRR_2024022806171  COMPLETE              45.77
nco_grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_timeoffset_suite_  COMPLETE              25.66
pregen_grid_orog_sfc_climo_20240228061718                          COMPLETE              17.97
specify_template_filenames_20240228061720                          COMPLETE              20.64
2019_hurricane_barry_20240228061722                                COMPLETE              39.50
----------------------------------------------------------------------------------------------------
Total                                                              COMPLETE             324.62

The final summary for Jet WE2E coverage tests:

----------------------------------------------------------------------------------------------------
Experiment name                                                  | Status    | Core hours used 
----------------------------------------------------------------------------------------------------
community_20240227181256                                           COMPLETE              20.36
custom_ESGgrid_20240227181323                                      COMPLETE              19.84
custom_ESGgrid_Great_Lakes_snow_8km_20240227181325                 COMPLETE              14.94
custom_GFDLgrid_20240227181330                                     COMPLETE              10.03
get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_nemsio_2021032018_202402  COMPLETE              14.00
get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_netcdf_2022060112_48h_20  COMPLETE              62.53
get_from_HPSS_ics_RAP_lbcs_RAP_20240227181357                      COMPLETE              17.86
grid_RRFS_AK_3km_ics_FV3GFS_lbcs_FV3GFS_suite_HRRR_20240227181403  COMPLETE             237.64
grid_RRFS_CONUS_13km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16_plot_20  COMPLETE              40.14
grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2_20240  COMPLETE               7.93
grid_RRFS_CONUS_3km_ics_FV3GFS_lbcs_FV3GFS_suite_RRFS_v1beta_2024  COMPLETE             509.09
nco_grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_HRRR_2024  COMPLETE              10.64
----------------------------------------------------------------------------------------------------
Total                                                              COMPLETE             965.00

Moving forward with merging this PR now.

ufs-community / ufs-srweather-app

[develop] Add three UFS Case Studies to WE2E testing process #1043