ufs-community / ufs-srweather-app

UFS Short-Range Weather Application
Other
56 stars 119 forks source link

[develop] Feature/metric update #890

Closed clouden90 closed 1 year ago

clouden90 commented 1 year ago

DESCRIPTION OF CHANGES:

Update the WE2E test used in srw_metric_example.sh to reflect the changes that the WE2E test "grid_SUBCONUS_Ind_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16" has been substituted with "grid_SUBCONUS_Ind_3km_ics_FV3GFS_lbcs_FV3GFS_suite_WoFS_v0" in #686 (The GFS v16 suite is not intended for use at such high resolutions, so using the WoFS_v0 suite was more appropriate in this case).

TESTS CONDUCTED:

To start, git clone the feature branch, then navigate to the ufs-srweather-app folder and proceed to check out the external repositories. Set the following environment variables: WORKSPACE (path to your ufs-srweather-app folder), SRW_PLATFORM (e.g., hera), SRW_COMPILER (e.g., intel), and SRW_PROJECT (e.g., nems). Once these are set, run the command ./.cicd/scripts/srw_metric_example.sh. This process is expected to take around 1 and a half hours to complete. Afterward, you should find an output file named skill-score.out that contains the skill score index. Tests have been conducted on Hera and Jenkins pipeline.

CHECKLIST

LABELS (optional):

A Code Manager needs to add the following labels to this PR:

MichaelLueken commented 1 year ago

The Cheyenne Intel tests were manually run on Hera and passed:

----------------------------------------------------------------------------------------------------
Experiment name                                                  | Status    | Core hours used 
----------------------------------------------------------------------------------------------------
custom_GFDLgrid__GFDLgrid_USE_NUM_CELLS_IN_FILENAMES_eq_FALSE      COMPLETE               7.65
grid_RRFS_CONUS_13km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16_plot     COMPLETE              29.78
grid_RRFS_CONUS_25km_ics_NAM_lbcs_NAM_suite_GFS_v16                COMPLETE              19.61
grid_RRFS_CONUScompact_13km_ics_HRRR_lbcs_RAP_suite_HRRR           COMPLETE              25.49
grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_RRFS_v1beta    COMPLETE               8.72
grid_SUBCONUS_Ind_3km_ics_HRRR_lbcs_HRRR_suite_HRRR                COMPLETE              16.69
pregen_grid_orog_sfc_climo                                         COMPLETE               7.46
specify_template_filenames                                         COMPLETE               7.88
----------------------------------------------------------------------------------------------------
Total                                                              COMPLETE             123.28

The Cheyenne GNU tests were manually run on Hera and passed:

----------------------------------------------------------------------------------------------------
Experiment name                                                  | Status    | Core hours used 
----------------------------------------------------------------------------------------------------
grid_CONUS_25km_GFDLgrid_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16      COMPLETE              20.45
grid_RRFS_CONUS_13km_ics_FV3GFS_lbcs_FV3GFS_suite_RRFS_v1beta      COMPLETE             227.87
grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v17_p8_plot  COMPLETE              27.59
grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_HRRR             COMPLETE              36.70
grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_RAP              COMPLETE              81.58
grid_RRFS_CONUScompact_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16   COMPLETE              23.08
grid_RRFS_NA_13km_ics_FV3GFS_lbcs_FV3GFS_suite_RAP                 COMPLETE             319.60
grid_SUBCONUS_Ind_3km_ics_NAM_lbcs_NAM_suite_GFS_v16               COMPLETE              51.59
MET_verification_only_vx                                           COMPLETE               0.12
specify_EXTRN_MDL_SYSBASEDIR_ICS_LBCS                              COMPLETE              10.36
----------------------------------------------------------------------------------------------------
Total                                                              COMPLETE             798.94

The manual test of MET_ensemble_verification_only_vx_time_lag on Hera GNU passed:

----------------------------------------------------------------------------------------------------
Experiment name                                                  | Status    | Core hours used 
----------------------------------------------------------------------------------------------------
MET_ensemble_verification_only_vx_time_lag                         COMPLETE               7.66
----------------------------------------------------------------------------------------------------
Total                                                              COMPLETE               7.66

The manual testing of the WE2E coverage tests on Jet (including custom_ESGgrid_Great_Lakes_snow_8km) have passed:

----------------------------------------------------------------------------------------------------
Experiment name                                                  | Status    | Core hours used 
----------------------------------------------------------------------------------------------------
community                                                          COMPLETE              18.71
custom_ESGgrid                                                     COMPLETE              18.53
custom_ESGgrid_Great_Lakes_snow_8km                                COMPLETE              19.11
custom_GFDLgrid                                                    COMPLETE              10.04
get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_nemsio_2021032018         COMPLETE              13.12
get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_netcdf_2022060112_48h     COMPLETE              60.19
get_from_HPSS_ics_RAP_lbcs_RAP                                     COMPLETE              17.25
grid_RRFS_AK_3km_ics_FV3GFS_lbcs_FV3GFS_suite_HRRR                 COMPLETE             241.34
grid_RRFS_CONUS_13km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16_plot     COMPLETE              41.43
grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2        COMPLETE               8.61
grid_RRFS_CONUS_3km_ics_FV3GFS_lbcs_FV3GFS_suite_RRFS_v1beta       COMPLETE             525.13
nco_grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_HRRR       COMPLETE              11.63
process_obs                                                        COMPLETE               0.30
----------------------------------------------------------------------------------------------------
Total                                                              COMPLETE             985.39

The Jenkins tests successfully passed on Gaea, Hera, and Orion, so I will now move forward with merging these changes.