billsacks
commented
5 years ago I realized that I think we could do this software testing with a single compset, even though that's not the way things are typically run: we could generate the cplhist data from a T compset rather than an I compset, then use a different case with the same T compset to run with the just-generated cplhist data. I'm not sure if we can expect them to be bit-for-bit the same, but we could at least run this as a smoke test to make sure that the basic machinery continues to work.
I'd like to have a test that demonstrates that these two things are bit-for-bit:
(1) Run an I compset test to generate cpl hist files to force a later T compset test
(2) Run a T compset test using those generated cpl hist data
Those two cases should be bit-for-bit for CISM (say, two years in). This would test both the generation of data in the coupler and the use of data in dlnd.
Doing this would require a mechanism to have a system test that compares two different compsets. I have a vague recollection that @ekluzek was working on something like this for testing cplhist-forced CLM cases (or maybe that such a thing was being developed to test ocean spinup?), but I can't find anything like that from a quick search. One possibility is to see if we can generalize system_tests_compare_two to support this: the main change would be replacing the create_clone call with a create_newcase call, with some specified compset to use for the second case. But if that adds too much complexity to system_tests_compare_two, we could just write a test outside of that framework.