search

ufs-community / ufs-srweather-app

UFS Short-Range Weather Application
Other
58 stars 119 forks source link

[develop] Update CMAQ task configs to use new shared conda configs on tier-1 platforms #684

Closed padhrigmccarthy closed 1 year ago

padhrigmccarthy commented 1 year ago

Description

Natalie Perlin created a new shared conda config to support CMAQ users on on Tier-1 platforms (Hera, Orion, Gaea, Jet, Cheyenne). This new environment is named 'regional_workflow_cmaq', and includes netcdf4 and dependencies (xarray, numpy, scipy). The task module configs for CMAQ in ufs_wrweather_app/modulefiles/tasks/HOST/ can now be updated to load this shared config when the tasks are run.

Solution

For each host, create a .lua file to load the 'regional_workflow_cmaq', and load that module for any tasks requiring the additional python modules. For example, on cheyenne we now have ufs_wrweather_app/modulefiles/tasks/cheyenne/miniconda_regional_workflow_cmaq.lua:

unload("python")
prepend_path("MODULEPATH","/glade/work/epicufsrt/contrib/miniconda3/modulefiles"
)
load(pathJoin("miniconda3", os.getenv("miniconda3_ver") or "4.12.0"))

setenv("SRW_ENV", "regional_workflow_cmaq")

And for the fire_emission task, we now have ufs_wrweather_app/modulefiles/tasks/cheyenne/fire_emission.local.lua:

load("ncarenv")
load("miniconda_regional_workflow_cmaq")

Alternatives

Alternatively, users can use a non-shared environment with the required python packages, and maintain their own local task scripts to manage configuration for tasks at runtime.

Related to

https://github.com/ufs-community/ufs-srweather-app/pull/656

chan-hoo commented 1 year ago

@MichaelLueken, I am reopening this issue because the path to a new python env needs to be updated on Hera and Orion too. I am too busy to do this right now. Once I complete my urgent issues for the nco implementation, I'll open a PR for this then.