ESMCI / cime

Common Infrastructure for Modeling the Earth
http://esmci.github.io/cime
Other
162 stars 207 forks source link

Reformat config machines #4542

Closed jedwards4b closed 11 months ago

jedwards4b commented 11 months ago

Reformats config machines so that the primary file contains only the regex field for machine name matching. The details for each machine are then read from files in machines/machinename/config_machines.xml This is v3 of config_machines.xml, this change is backward compatible with v2.

Test suite: scripts_regression_tests.py Test baseline: Test namelist changes: Test status: bit for bit

Fixes

User interface changes?:

Update gh-pages html (Y/N)?:

jedwards4b commented 11 months ago

Added the config_machines_version3.xsd and the code to pass both schemas in and check the one based on the file version.

rljacob commented 11 months ago

This is welcome since config_machines.xml can become a giant file. Does the code in the per-machine files look just like the XML blocks in the current combined file?

jedwards4b commented 11 months ago

@rljacob I made it backward compatible so no changes are required in e3sm. Yes the format of individual files is nearly the same as the current block - the only difference is that the NODENAME_REGEX has been moved out of the block to the top level file. Here is a link to the reformated cesm machines files.

rljacob commented 11 months ago

Is there a way to do one subdir with a uniquely named file per machine instead of a uniquely named subdir for each machine?

jedwards4b commented 11 months ago

What is your argument for that change? I considered it, but plan to treat config_batch.xml in the same way - so there will be a config_batch.xml file in each machine subdir.

rljacob commented 11 months ago

If there's going to be more then one file in each subdir, then it makes more sense. Does the subdir name have to match the string you pass to --machine ?

jedwards4b commented 11 months ago

Yes, currently the subdirectory name is expected to match the string in MACH=

jedwards4b commented 11 months ago

The cprnc submodule does seem to be out of sync with master, my attempts to fix the issue have failed - @jgfouca can you have a look? There should not be any changes in cprnc due to this PR.

jgfouca commented 11 months ago

@jedwards4b , I just merged https://github.com/ESMCI/cime/pull/4545 , which should put us on the latest cprnc main commit. You should rebase or upstream this branch, take the cprnc commit that's on master, and the conflict will resolve.

jedwards4b commented 11 months ago

@jgfouca I just did a rebase again - it still doesn't seem to have worked.

jgfouca commented 11 months ago

It looks like the conflict is gone?

jgfouca commented 11 months ago

Oh, I see this PR is still trying to change the submodule. Try this: git checkout origin/master $path_to_cprnc ; git add $path_to_cprnc; git commit. This assumes origin is the main repo, not your fork.

jedwards4b commented 11 months ago

Still no help: git checkout origin/master CIME/non_py/cprnc ; git add CIME/non_py/cprnc; git commit Updated 0 paths from b6c797123 Check Xml............................................(no files to check)Skipped Fix End of Files.....................................(no files to check)Skipped Trim Trailing Whitespace.............................(no files to check)Skipped black................................................(no files to check)Skipped pylint...............................................(no files to check)Skipped On branch reformat_config_machines Your branch is up to date with 'mydev/reformat_config_machines'.

Untracked files: (use "git add ..." to include in what will be committed)

DiagsCase.py

#testcase.py#
CIME/#test_status.py#
CIME/.#test_status.py
CIME/ParamGen/requirements.txt
CIME/case/#check_input_data.py#
DiagsCase.py
ccs_config/
scripts/create_newcase.py.log
scripts/create_test.log
scripts/create_test.out
scripts/create_test2.out
scripts/test.out
testcase.py

nothing added to commit but untracked files present (use "git add" to track) derecho5: ~/sandboxes/cime :) git push mydev Everything up-to-date

jgfouca commented 11 months ago

@jedwards4b , can you push this branch to the main repo? I'd like to check it out and modify it.

jedwards4b commented 11 months ago

I could but I shouldn't have to, you can clone or add my fork with git remote add

jgfouca commented 11 months ago

I checked out your branch and somehow your cprnc submodule is still set to an older commit. This should fix it:

git fetch $esmci_remote
git cherry-pick $esmci_remote/jgfouca/jime_branch
git submodule update
git push
jedwards4b commented 11 months ago

That did it - thank you!