NOAA-EMC / global-workflow

Global Superstructure/Workflow supporting the Global Forecast System (GFS)
https://global-workflow.readthedocs.io/en/latest
GNU Lesser General Public License v3.0
70 stars 161 forks source link

Enhance and simplify the setup scripts and resource configs to handle multiple RUNs' #2693

Open DavidHuber-NOAA opened 1 week ago

DavidHuber-NOAA commented 1 week ago

What new functionality do you need?

As a followup to https://github.com/NOAA-EMC/global-workflow/issues/177, the setup scripts, config.resources, and possibly other config/env files should be improved to generate resources based on the appropriate CDUMP(s) (gfs, gdas, enkfgdas, enkfgfs, gefs, and soon sfs and possibly enkfsfs) to more cleanly assign resources for a given experiment. Additionally, the resource variables should be exported at the end of config.resources as simple variables (e.g. instead of npe_nodes_<job>_<CDUMP>, export npe_nodes and pick up on these new variables in each of the env files.

What are the requirements for the new functionality?

Each job's resources would be defined at setup time and runtime in a simple manner.

Acceptance Criteria

Suggest a solution (optional)

No response

DavidHuber-NOAA commented 4 days ago

This will go hand-in-hand with #1299, so I am tackling both at once.