ClimateMachine needs a more comprehensive set of experiments with verifiable output

[thomasgibson]

Description

Just summarizing part of the conversation over today's spring meeting.

I think all of us came to the conclusion that verifying the output of the LES and GCM drivers right now is incredibly difficult. There are a number of reasons for this, one being related to diagnostic output (i.e. not having easy access to perturbation fields or vorticity profiles). Right now, it's hard for many of us to look at the prognostic variables (like total energy for instance) for a particular experiment and determine dycore efficacy. That's a separate issue being addressed (Can someone maybe provide a reference for discussion related to revamping diagnostic output? There are multiple issues and I'm not sure which to reference.)

The main purpose of this issue is to help organize a list of desirable dycore experiments we can use to verify the efficacy of the code (in particular, the dycore and dycore + physics).

This should be an open discussion for all to participate!

Here is a link that I think we should seriously consult:

• https://earthsystemcog.org/site_media/docs/DCMIP-TestCaseDocument_v1.7.pdf (2012 DCMIP)
• https://github.com/ClimateGlobalChange/DCMIP2016 (Paul Ullrich's public repo for 2016 DCMIP, which includes documentation of test cases)

For those less familiar with DCMIP, or the Dynamical Core Model Intercomparison Project, is a well-documented effort to design a standard set of test suites for atmospheric dynamical cores of varying complexity. This will allow to confidently verify the model output against other atmosphere models.

A nice set of standard experiments collected together for dycore benchmarking in LES mode (i.e. 2D and 3D limited area) can be found in the following references:

• 2D and 3D rising thermal bubble
• Density current
• Hydrostatic and non-hydrostatic mountain gravity waves

@phdthesis{restelliPHD2007,
   author = {{Restelli},M.},
   title = {Semi-{L}agrangian and semi-implicit discontinuous {G}alerkin methods for atmospheric modeling applications},
   school = {Politecnico di Milano},
   year  = {2007},
}

Note: If you can't find this thesis online, ask Simone for a copy.

• Hydrostatic and non-hydrostatic mountain gravity waves

• 2D Inertia-gravity wave in a periodic channel

• Scalability tests on multi-core CPU

  @article{giraldoRestelli2008,
      author = {F. X. Giraldo and M.. Restelli},
  journal = {J. Comput. Phys.}
  title = {{A study of spectral element and discontinuous Galerkin methods for the Navier–Stokes equations in nonhydrostatic mesoscale atmospheric modeling: Equation sets and test cases}},
  url = {https://doi.org/10.1016/j.jcp.2007.12.009},
  year = 2008,
  volume = {227},
  number = {8},
  pages = {3849-3877},
  }
  @article{kellyGiraldo2012,
      author = {J. F. Kelly and F. X. Giraldo},
  journal = {J. Comput. Phys.}
  title = {{Continuous and discontinuous Galerkin methods for a scalable three-dimensional nonhydrostatic atmospheric model: Limited-area mode}},
  url = {https://doi.org/10.1016/j.jcp.2012.04.042},
  year = 2012,
  volume = {231},
  number = {24},
  pages = {7988-8008},
  }

• Baroclinic-wave in a 3D channel with geostrophically balanced background

• Non-linear advection (i.e. Burgers equation)

• 2D and 3D rising thermal bubble

• Density current

  @article{marrasNazarovGiraldo2015,
  author = {{Marras},S. and {Nazarov}, M. and {Giraldo},F.~X.},
  title = {{Stabilized high-order Galerkin methods based on a parameter-free dynamic SGS model for LES}},
  journal = {J. Comput. Phys.}, 
  volume = {301},
  pages = {77-101},
  year = {2015},
  },

Additional context

Add any other reasons why this should be addressed.

For CLIMA Developers

• [x] This feature can be added (if it cannot be, explain why in a comment below -- lack of technical expertise, not relevant to the scope of this project, too ambitious)
• [ ] There is a timeline for when this feature can be implemented
• [ ] The feature has been (or will be) implemented (Please link the PR)

[thomasgibson]

This also applies to the ocean dycore as well. So I think it's probably a good idea to do similar verification. I am less familiar with the ocean literature, so I'm going to need some oceans guys to chime in with possible references for standard ocean dycore tests.

[szy21]

For comparison, the results from all the models participating in DCMIP 2012 are here: https://www.earthsystemcog.org/projects/dcmip-2012/.

[smarras79]

For LES modality testing, the code currently contains the following LES tests that are accepted as standard tests to assess the dynamical core:

• tutorials/Atmos/risingbubble.jl --> complete. Currently PR 1127 possibly merging soon.
• tutorials/Atmos/densitycurrent.jl --> complete. Currently PR 1151 in review.

I set and tested these two benchmarks to run to completion (timeend = 1000 s and timeend = 900 s, respectively) in only ~5 minutes of wall-clock on 1 GPU on central. This will permit these tests to always be run with ease when changes across the code are pushed.

We are still missing:

• hydrostatic and non-hydrostatic gravity wave tests --> I will take care of this.

Moist dynamics: a simple precipitating moisture with a dynamically simple problem to solve is

• 3D squall line. This problem can be executed with a relatively coarse resolution using fully explicit time integration. A setup that I typically use (@slifer50 has been running it lately in diffrent configurations) allows a complete run within ~20 minutes of wall clock on 1 GPU.

Notice: Dycoms and Bomex are NOT standard benchmarks. We can use them, but a coarse resolution for either one requires > 30 minutes of wall clock to reach completion.

[blallen]

@christophernhill and @jm-c, is there anything like DCMIP for ocean models? I know we are waiting on split-explicit and implicit diffusion before we can make the test suite feasible, but it would be good to know what we are aiming for.