Detect if preview_namelist needs to be called and avoid calling it.

[rljacob]

preview_namelist is called multiple times between "create_newcase" and "case.submit" to make sure user changes make it in to the run, even if users haven't made any changes. Preview_namelists can be expensive. After the first call to preview_namelists, add logic to check if any files have been modified that require preview_namelists to be run again and, if not, don't.

[jedwards4b]

File to be checked include: usernl envrun.xml user_streams.txt

[billsacks]

I like the idea of speeding things up in this way. But I'm also worried that this could result in some subtle bugs if we forget to add some file to the list of possible dependencies. I'm especially worried about what happens in the future when someone adds a new dependency (maybe one that just applies to certain model configurations) but forgets to update the dependency list. This could lead to some really bad bugs in that they'd only turn up with a certain very specific order of operations, and they'd cause the model to behave subtly differently from what you expected.

So if we do this, I think it's worth putting some thought into how we could either:

1. Set up the code to prevent the above situation, or

2. Set up a test that would fail if some necessary file is left off of the dependency list

One possible approach towards (1) would be: The high-level preview_namelist code explicitly hands down the listed dependent files to the implementation code below it. The lower level code is somehow prevented from accessing anything else in the case directory. (I'm not sure how feasible that is; it's just what popped to mind.)

[rljacob]

This is a duplicate of #905

[mfdeakin-sandia]

If this is a duplicate, can it (or the other one) be closed? This one does seem more informative, though the other one is assigned

[cameronsmith1]

A radical suggestion: Why call preview_namelist at all?

There is a common problem where users don't realize how it all works, and look at a namelist from part-way through the setup process and get confused, eg https://github.com/ACME-Climate/ACME/issues/1539

The only purposes I can see for calling preview_namelist are (both of which are rare): 1) if a user wants to setup a new run and wants a draft namelist and doesn't have an existing run to start from, 2) someone is trying to debug CIME.

We can avoid the confusion and still satisfy (1) & (2) if a) eliminate preview_namelist from CIME calls, except for the final call when the model is run, b) if a user wants to preview the namelist (eg 1 & 2 above) they can run preview_namelist manually.

[jgfouca]

@cameronsmith1 I originally thought the same thing myself. If preview_namelists is just setting up input files for running the code, why would it need to be called except for once right before the run? Well, turns out certain components depend on namelists in order to be able to build themselves.

Basically, we didn't manage how namelists should be used be components. They now represent a state bucket that can be accessed or changed at any time by any component.

[cameronsmith1]

Wow. I thought that is what the XML files were for. Doing it through the namelists seems dangerous, since the namelist is often changed by the user, so consistency and provenance will be lost.

[jgfouca]

The namelist situation is probably the biggest flaw in CIME right now. It's not an easy thing to fix either, because it involves changing how components in both ACME and CESM use CIME.

[cameronsmith1]

Is there a list of the namelist variables which get used in this way (so users can avoid messing them)?

[jgfouca]

@jedwards4b may know.

[jgfouca]

The performance of preview_namelists has improved dramatically as scripts have switched to python. This is still something we want to think about longterm; to reduce the tight coupling between namelists and XML files and what the correct usage of CIME is within component nml/build scripts.