[SRW-AQM] AQM sample script 'config.aqm.community.yaml' fails due to missing data files

[chan-hoo]

Expected behavior

The provided sample script for SRW-AQM is completed successfully.

Current behavior

The fire_emission task fails due to missing data files:

202302170000           fire_emission                    55687737                DEAD                   1         2          18.0

+ print_msg.sh[275]: err_msg='\nFire Emission RAW data does not exist:
  FILE_13km_md1 = "Hourly_Emissions_13km_202302152300_202302152300.nc"
  DCOMINfire = "/scratch2/NCEPDEV/naqfc/RRFS_CMAQ/RAVE_fire"'
+ print_msg.sh[278]: printf 'FATAL ERROR:

Machines affected

Hera

Steps To Reproduce

1. Run the sample script config.aqmcommunity.yaml.

[MichaelLueken]

@chan-hoo - I have made changes in a local copy of the SRW App to use the config.aqm.retro.hera.yaml settings in the current config.aqm_grid_AQM_NA13km_suite_GFS_v16.yaml WE2E test configuration. That work can be found in /scratch2/NAGAPE/epic/Michael.Lueken/ufs-srweather-app.

The modified files are:

• modulefiles/tasks/hera/aqm_lbcs.local.lua - Loading the new BUILD_CONDA environment load("python_srw_cmaq")
• parm/wflow/aqm_prep.yaml - native: entry for task_nexus_gfs_sfc: to set -n 1 for service partition jobs
• scripts/exregional_fire_emission.sh - Update ${DCOMINfire}/${aqm_fire_file_fn} entries to remove ${yyyymmdd}
• tests/WE2E/test_configs/aqm/config.aqm_grid_AQM_NA13km_suite_GFS_v16.yaml - Updated to use entries from config.aqm.retro.hera.yaml
• ush/machine/hera.yaml - Add SCHED_NATIVE_CMD_HPSS for service partition and update paths to AQM fix files.

[chan-hoo]

@MichaelLueken, thanks! I'll put them into my PR.

[chan-hoo]

@MichaelLueken, I'd like to change the dates in the sample script to:

  DATE_FIRST_CYCL: '2023111000'
  DATE_LAST_CYCL: '2023111100'

My concern is that the GFS netcdf files are not provided in the staged data directory for WE2E. Can you let me know where I can check this?

[MichaelLueken]

@chan-hoo, in the ush/machine/*.yaml files, the entry:

TEST_EXTRN_MDL_SOURCE_BASEDIR

shows the directory that holds the staged ICs and LBCs data on a given platform.

You are correct that there is no GFS netcdf files staged on any of the Tier-1 RDHPCS machines for 2023111000 nor 2023111100. I was only able to make my changes work on Hera (and Jet) by removing the USE_USER_STAGED_EXTRN_FILES entries.

We will need to work with the SI team to get 2023111000 and 2023111100 files staged on other machines.

[chan-hoo]

@MichaelLueken, Thanks! Who can I ask for this in the SI team? Ratko may be in SI. Can I ask him?

[MichaelLueken]

@chan-hoo, Ratko and Natalie are both members of the SI team who might be able to assist with adding the updated GFS netcdf files to the staged data across the EPIC maintained platforms. At the very least, they will be able to either forward the request to those who can assist if they are unable to.

[chan-hoo]

Hi, @RatkoVasic-NOAA @natalie-perlin , could you please copy the following directories to the staged-data directories for WE2E on the tier-1 machines?

• From:

  /scratch2/NCEPDEV/fv3-cam/Chan-hoo.Jeon/UFS-SRW-AQM/expt_dirs/fv3gfs_netcdf/2023111000
  /scratch2/NCEPDEV/fv3-cam/Chan-hoo.Jeon/UFS-SRW-AQM/expt_dirs/fv3gfs_netcdf/2023111100

• To (for example, on Hera):

  /scratch1/NCEPDEV/nems/role.epic/UFS_SRW_data/develop/input_model_data/FV3GFS/netcdf/

  These netcdf files are necessary for the new WE2E test of AQM.

[RatkoVasic-NOAA]

@chan-hoo here is the list of machines:

• [x] Hera
• [x] Gaea
• [x] Jet
• [x] Orion
• [x] Hercules
• [x] Derecho
• [ ] Cloud? I'll check boxes when done.