Is your feature request related to a problem? Please describe.
For production work, scientists would like to be able to run bronx XMLs (and yamls, now that we can make them) in the new canopy workflow. If we combine together the new yaml structure/command grouping and the-tool-currently-known-as-fre-pp-wrapper, we can get most of the way there.
Describe the solution you'd like
This divides into 6 (maybe 5) tasks:
[ ] Modify the pp starter for freVersion != Bronx
[ ] Remove custom platform csh from the xmls
[x] Keep the am5 yaml include and the attached directory structure
[x] Main yaml has 3 sections
[x] configure_yaml now loads 2 yamls
[ ] offline conversion of xml to yamls in a separate tool
Modify the pp starter such that if the freVersion is not bronx, then do some different stuff in pp.starter:
Swap out "-x am5.xml" with "-c $(includeDir)/am5.yaml"
Do not retrieve the history and ptmp directories using frelist, and do not stage the history files to PTMP.
Custom platform csh is not needed in the XMLs any longer.
User specify that they want to use canopy in the section as is allowed today:
canopy
3. FRE Bronx will copy over all the files in $(includeDir), so we'll keep the am5.yaml and the yaml_include files underneath it. e.g.
- am5.xml
- xml_include/am5.yaml
- xml_include/yaml_include/pp.c96.yaml
- xml_include/yaml_include/pp.c192.yaml
4. main yaml has 3 sections going forward:
- properties: user-defined variables that get expanded in directories and experiments sections
- directories: output from pp
- experiments: a set of keyed pairs that contain 2 things: a name (experiment name) and an experiment yaml
5. configure_yaml needs to load 2 yamls - the main and an experiment yaml that is referred to in the main yaml
6. We need an offline tool for converting XML -> YAML but there's not much urgency on that - for now, we can hand-edit and check to make sure XML and YAML properties match.
**Additional context**
This is modified from an email chain sent out on 2022-07-08
Describe the solution you'd like
This divides into 6 (maybe 5) tasks:
Do not retrieve the history and ptmp directories using frelist, and do not stage the history files to PTMP.