Port single column test group

[cbegeman]

This PR ports the ocean/single_column_model/planar/cvmix_test from compass-legacy as ocean/single_column/10km/cvmix.

The following changes are made from compass-legacy:

• creating the initial state in python instead of MPAS-Ocean's init mode
• using a uniform vertical grid rather than a custom grid, which was specified in init mode
• adding a viz step

Checklist

• [x] User's Guide has been updated
• [X] Developer's Guide has been updated
• [X] API documentation in the Developer's Guide (api.md) has any new or modified class, method and/or functions listed
• [X] Documentation has been built locally and changes look as expected
• [X] Testing comment in the PR documents testing used to verify the changes
• [X] New tests have been added to a test suite

[cbegeman]

Testing

I have run this test on chrys with intel, openmpi and visually inspected the results. I verified that the forward namelist and streams files match between legacy and polaris.

[cbegeman]

@xylar Thanks for the review. I'll make those changes.

W.r.t. the state validation failure, I'll try to run without the change in horizontal resolution from legacy.

[xylar]

I was able to reproduce the NaN issue on Chrysalis with intel, though it happens at day 11, rather than day 6:

/lcrc/group/e3sm/ac.xylar/polaris_0.1/chrysalis/test_20230426/pr_cvmix/ocean/single_column/10km/cvmix/forward

[cbegeman]

@xylar I finished a 20-day run on chrys with intel, openmpi at 960km resolution. When you have a chance, can you try on linux with gnu again at 960km? You probably saw that you just change the resolution argument at https://github.com/E3SM-Project/polaris/blob/89224ea2fae2a44430a0cd494bf71134e16c1807/polaris/ocean/tests/single_column/__init__.py#L16

I'm troubled by the fact that this test crashes at 10km. It just makes sense to me to run it at something closer to typical resolutions. The init mode routine was written by Todd. @vanroekel do you have any ideas about why this cvmix test (formerly in legacy) would have been set up at such coarse resolutions and why it might crash at 10km resolution?

[cbegeman]

The differences between the config options for this test and the defaults in https://github.com/E3SM-Project/E3SM/blob/3e1e3db133aa17b9bd53003cc5ffa3de6790901f/components/mpas-ocean/bld/namelist_files/namelist_defaults_mpaso.xml:

[first option is the one in this test, second one is from defaults]

config_cvmix_kpp_matching: 'MatchBoth', 'SimpleShapes'
config_cvmix_kpp_interpolationOMLType: 'cubic', 'quadratic'

I think it makes sense to match the current defaults. I tested whether this would allow us to run at 10km and it failed even earlier on chrys, intel: at day 2 instead of day 11.

[xylar]

@cbegeman, yes, at 960 km horizontal resolution, I'm able to run for 20 days without a problem.

[cbegeman]

@xylar I suppose since no insights are forthcoming we should just proceed with the 960km resolution and the original config options so it's in place for the hackathon. Do you agree?

[xylar]

@cbegeman, I tried shortening the time step to 1 minute (btr_dt of 3 seconds) and the crash at 10 km happened even sooner so time step doesn't see to be the fix.

[xylar]

@xylar I suppose since no insights are forthcoming we should just proceed with the 960km resolution and the original config options so it's in place for the hackathon. Do you agree?

Yes, I don't see any harm. Parameters that allow a finer resolution could be investigated at the hackathon if it feels like a good use of time.

[cbegeman]

@xylar Thanks for your feedback and testing!