xylar
commented
1 year ago Testing
I ran all ocean test cases except the 1 and 4 km RPE tests, comparing against #111 as a baseline. All tests passed except the manufactured solution test, which is currently failing on purpose to indicate a lower than desired order of convergence in MPAS-Ocean.
This merge simplified how validation is handled in Polaris.
Baseline validation is handled by passing an optional argument,
validate_vars, to the methodStep.add_output_file(). The variables are a list of variables to compared with the same output file in a baseline, if one is provided.Entirely separate from baseline comparison is the comparison between outputs in two steps. This is performed by adding a third step (e.g.
validate) that compares the outputs and raises an error if they don't match. Such a validate steps has been added to baroclinic channel test cases and the sea-ice component's exact restart test.The
validate()method has been removed from test cases.For now, the unused capability to evaluate timers has been removed. This will be revisited once test cases with the need arise, but the current capability was clumsy enough to be better purposefully revisited rather than propagated onward by default.
Checklist
api.md) has any new or modified class, method and/or functions listedTestingcomment in the PR documents testing used to verify the changesBased off of #111, which is based off of #110, which is based off of #112. (Yes, this is getting out of hand!)