Initial runscript framework

[mkavulich]

Expected Differences

• [x] Do these changes modify the system output in any way? Yes

This PR is the initial "skeleton" framework for the eventual run scripts for creating, running, and monitoring an end-to-end NWP workflow for the wrfcloud framework. An initial plan was outlined in the design document; here is a summary of this initial version:

Initial framework summary

./run.py is the main runtime script. This python script takes as input a single command-line argument, which is the "name" of the run. This should be unique; it could be of the format run_{user/group}_{unix_datetime} or something similar. The ./run.py script will take this argument, and use it to read in an experiment configuration file, {name}.yml. The configuration file will contain all the user-specified information needed for a particular run.

If no name is provided, the script will run a test case, named "test" and using the provided example config file test.yml.

As the run goes on, all work will be done in a working directory, which for now is simply a subdirectory named {name}, the name of the run as described above.

Inside of ./run.py, the main function does very little directly, except to set up the RunInfo object that contains all the run-specific information, much of which is read directly from the run configuration .yml file. Other than that it mostly just calls other functions to get the job done.

• setup_logging
  • Sets up logging. Logging is handled via the python builtin logging module; output is handled by printing statements with calls to logging.LEVEL('Message'), where LEVEL is one of the logging levels as defined in the documentation. All messages of level debug or higher will be printed to a log file named debug.log in the experiment directory
• ungrib.main
  • The main function of the ungrib module, contained in ungrib.py and responsible for running the ungrib task. This is built out a little, and currently creates a namelist and links in some input files; I will continue to work on this part of the run going forward.
• metgrid.main
  • The main function of the metgrid module, contained in metgrid.py and responsible for running the metgrid task. This is currently mostly a placeholder, though it does call the same function to create the namelist.
• real.main
  • The main function of the real module, contained in real.py and responsible for running the real.exe task. This is currently a placeholder.
• wrf.main
  • The main function of the wrf module, contained in wrf.py and responsible for running the wrf task. This is currently a placeholder.
• postproc.main
  • The main function of the postproc module, contained in postproc.py and responsible for running the postprocessing tasks. This is currently a placeholder; may need to be renamed and/or broken down into multiple modules in the future depending on design choices.

Finally, there are two subdirectories:

• domains/
  • Contains a directory for each experiment configuration (a specific domain setup). Currently there is just one "test" configuration, which contains a template namelist.wps that is modified for a particular experiment's parameters. Eventually there will be a namelist.wrf as well, and it will contain static data files on disk (though not in this repository) such as geo_em files.
• tools/
  • Contains modules for tools that may be of use by multiple main tasks/modules. Currently there is just one tool, for creating the WPS namelist.

Other resources

A .yml environment file for building a conda environment containing all the packages necessary for this initial version has been added in wrfcloud/python/src/wrfcloud/resources. This includes pylint, which is recommended to be used to check the python code meets all best practices/standards as we move forward building the system.

Pull Request Testing

• [x] Describe testing already performed for these changes:

  • This set of scripts and modules does not do much yet, but I have tested the build-in test case on Cheyenne and it creates a test/ungrib directory containing links to the provided geo_em and GRIB input files, as well as creating a namelist.wps based on the user's settings.

• [ ] Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:

  • Suggestions for tests welcome!

• [ ] Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? Yes

  • I have attempted to make the code as self-documented as possible, let me know if you notice any shortfalls.

• [ ] Do these changes include sufficient testing updates? ??

• [ ] Will this PR result in changes to the test suite? ??

• [ ] Please complete this pull request review by 2022-07-21

Pull Request Checklist

• [ ] Review the source issue metadata (labels, project, and milestone).
• [ ] Complete the PR definition above.
• [ ] Ensure the PR title matches the feature or bugfix branch name.
• [ ] Define the PR metadata, as permissions allow. Select: Reviewer(s)
  • Done Select: Project
  • Do we have one? Select: Milestone as the version that will include these changes Select: Development to link to the original development issue.
• [ ] After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
• [ ] Close the linked issue and delete your feature or bugfix branch from GitHub.

[mkavulich]

@hahnd Thanks again for your input, and thanks for running the changes through pylint since I forgot to on the last few commits! All the changes look good, and now I know about type and return hints for future use :)