search

MPAS-Dev / compass

Configuration Of MPAS Setups
Other
10 stars 37 forks source link

Prioritized list of test cases to port to the new compass package #51

Open xylar opened 3 years ago

xylar commented 3 years ago

Here is a list of test cases from legacy COMPASS that could be ported to the new compass python package. I would like help putting them in order of priority.

A checked box below means the test case has already been ported. Strike-through means we don't plan to port them.

legacy COMPASS landice test cases to port:

Land-ice tests not yet slated for porting:

legacy COMPASS ocean test cases to port:

Ocean tests not yet slated for porting:

Ocean tests that will not be ported:

xylar commented 3 years ago

@vanroekel, @matthewhoffman, @sbrus89, @mark-petersen, @milenaveneziani, @darincomeau, could you all help me prioritize these test cases. If you want to prioritize them, move them up the list either by dragging them or editing the comment. If you know a test case isn't needed (it will still be available in legacy COMPASS, it just won't be ported), please use ~~ to add strike-through.

Please @ anyone else whose feedback we also need.

xylar commented 3 years ago

@trhille, you may also be a good person to give feedback on the landice list.

vanroekel commented 3 years ago

Thanks for making this list @xylar. I've made a few minor shifts. I've also bumped cvmix to the bottom as I'd like to take these on myself for porting to the new compass package.

@mark-petersen if you don't think the redi verification tests are particularly helpful to have for standard testing, please de prioritize those.

xylar commented 3 years ago

@vanroekel, I've moved the cvmix test cases up in priority but put your username after them, I hope that's okay. This way we can kind of keep track of who's planning to do what.

vanroekel commented 3 years ago

great idea. Thanks for making that change.

sbrus89 commented 3 years ago

@xylar, I'd like to take on the hurricane/tides cases (I'll mark them in the comment). That way I can learn the ins and outs of the new COMAPSS. It'll also be a good opportunity for me to possibly prune these as necessary when I go through them. Thanks again for doing all this!

xylar commented 3 years ago

Thanks @sbrus89. I'm happy to help. I'd recommend holding off on starting your porting until #28 gets merged. Perhaps in a couple of weeks.

sbrus89 commented 3 years ago

@xylar, sounds good. Thanks for the heads up. I'm excited to start using the new compass!

matthewhoffman commented 3 years ago

@xylar , I've updated the landice lists. There may be additional round of decisions on what to port when (or at all), but this narrows things down a bit.

xylar commented 3 years ago

Thanks @matthewhoffman, that's really helpful. I'll work my way through the ones with my name on them in the next week or 2.

xylar commented 2 years ago

@cbegeman and @dengwirda, if it makes sense to port some of the ocean/dam_break and ocean/drying_slope test cases from legacy to master, please list the tests we want to port and I can move them to the "to do" kind of section above with some names (including mine).

cbegeman commented 2 years ago

@xylar Thanks. It doesn't appear that ocean/drying_slope was based on a standard test case but I think it could be made to resemble the simplest 1-d uniformly sloping basin test case from this paper https://doi.org/10.1016/S0378-3839(98)00015-5. It's worth considering the ocean/dam_break test case but let's make sure we can pass the simplest wetting-and-drying test case first.

xylar commented 2 years ago

Okay, rather than trying to port anything from ocean/drying_slope, it might be best to start fresh with a new test-case name and such.

sbrus89 commented 2 years ago

@xylar, the drying slope and dam break cases are both taken from this paper: Warner et al. 2013 which describes the wetting and drying implementation in ROMS. I believe both test cases use the ROMS data in the analysis steps for comparison. The MPAS results for these cases were published here: Cao et al. 2021

sbrus89 commented 2 years ago

Which is to say that I think carrying these cases over (rather than starting fresh) is important. Of course, adding additional cases would be great too.

sbrus89 commented 2 years ago

Also, I happened to be re-running the drying slope case the other day and noticed that it no longer behaves the way it originally did. My results from the other day look like this: dryingslopecomparison

versus the result from the Cao et al. 2021 paper:

Screen Shot 2022-02-15 at 2 18 02 PM

Note that the 0.05 days result is way off in the top plot.

dengwirda commented 2 years ago

@xylar @cbegeman I think another (new) test that could be useful as a wetting + drying verification is the oscillations-in-a-bowl case, similar to that described in this NEMO paper: Implementation and assessment of a flux limiter based wetting and drying scheme in NEMO. With repeated wetting + drying cycles, this should give us a way to test the symmetry of the scheme, and any spurious frictional losses over time. A single-layer MPAS-O tides case (with small initial min.-depth) could also be a good 'do-no-harm' shallow-water case, to ensure wetting + drying doesn't degrade deep ocean response.

cbegeman commented 2 years ago

@sbrus89 Great! I'm happy to hear that both of these test cases derived from existing tests. Then they do seem worth bringing over.

@xylar Let me know how you'd like to share the work and @dengwirda whether you'd like to get your hands dirty with e.g. the bowl case.

xylar commented 2 years ago

@sbrus89 and @cbegeman, could we check off which of these are priorities?

First round:

Second round:

Third round:

Later (if at all):

Do you have permission to check/uncheck these? If not, comment and I'll do it.

xylar commented 2 years ago

To get the ball rolling, I have checked the ones I would be inclined to port. It's typically easy to handle multiple resolutions in one port. It shouldn't be too hard to handle multiple vertical coordinates in one port.

@cbegeman made clear that the dam-break test case would be a later priority because it's more rigorous and challenging.

cbegeman commented 2 years ago

@xylar I can't edit but I think what you've designated is fine. Since we're starting with only 1 vertical level, it's pretty low priority to handle multiple vertical coordinates.

xylar commented 2 years ago

@cbegeman, good point. I'll prioritize sigma, since that seems like the most sensible coordinate for a single layer.

xylar commented 2 years ago

@sbrus89, there are many Hurricane test cases slated for porting (see above) but I didn't see an exact match to the ones you created. Could you check off whichever ones you ported (or the closest matches). If you don't plan to port some of those tests after all should we move some to "won't port"?

sbrus89 commented 2 years ago

@xylar, I just did what you suggested.

xylar commented 2 years ago

Excellent, much appreciated @sbrus89!