Organize non-default options into for-testing physics options

[billsacks]

Thinking about how to test multi-parameterization parts of the code (i.e., where there are a number of different options), I'm coming to feel that we should try to cover most or all of the options via system tests – in particular, via ERP tests, which cover exact restart and changing processor count, because these can catch bugs that are hard or impossible to catch via unit tests. (Changing processor count can catch things like accidentally assigning to a whole array when you mean to assign to a single element, or having an index variable in one loop accidentally using its value from the last iteration from the previous loop because you forgot to set it in the current loop.) (We could have a long-ish ERP test combined with a short SMS_D test, or just a short-ish ERP_D test.)

Currently, we have a ton of testmods, most of which just change one thing. To make it feasible to test each option of each parameterization, I'd like to consider introducing new "physics options" that are used only for testing.

For example, we could have test_option1, test_option2, etc. This could be done with testmods, but it's probably better to do it in namelist_defaults: for a given variable that controls different options, we'd make sure that all possible values fall under some physics option that's tested – if not one of the "real" options, then one of these test options. Then we'd have system tests covering each of the physics options, including these test options.

So physics "test_option1" would choose a suite of options that are all the defaults for test_option1, etc. These would be arbitrarily mixed and matched; some care may be needed to ensure that we don't try to combine incompatible options.

For this to work, I think we'd need a default in namelist_defaults if there's no match – e.g., what happens for test_option3 for something that only has two options.

[billsacks]

An example of what I'm thinking about is the irrig_alternate testmod that I introduced in 68eb76c32: rather than having a test that just differs in this minor way, we could combine this with testing some other non-default options.

[billsacks]

For (unusual) options like glacier region behaviors, where there can be a different option for each region: If all behaviors are already included by default for at least one region, then we're probably fine; if not, then we could have, e.g., test_option1 sets the behavior to X (where X is some otherwise-untested behavior) in all regions (i.e., all array elements).

[ekluzek]

This should be thought about in terms of #585