NOAA-GFDL / MOM6

Modular Ocean Model
Other
27 stars 59 forks source link

Model dies with segmentation fault if depth greater than 9250 m #491

Closed pittwolfe closed 5 months ago

pittwolfe commented 1 year ago

The model dies with a segmentation fault if the ocean depth is anywhere greater than 9250 m unless non-obvious, non-default settings are used.

Reproduction

Run the "double_gyre" example out of the box, except set TOPO_CONFIG = "flat" and MAXIMUM_DEPTH to anything greater than 9250.

Error message

Depends on fortran runtime and optimization level. Optimized Intel is unintelligible. gfortran helpfully produces

Error termination. Backtrace:
At line 465 of file ../../../../src/MOM6/src/ALE/MOM_regridding.F90
Fortran runtime error: Index '41' of dimension 1 of array 'woa09_dz' above upper bound of 40

Expected behavior

No segmentation faults. If the parameters are inconsistent, model should post an error message and exit gracefully.

Origin

initialize_regridding, called by the diagnostics mediator, will default (for convenience) to WOA09 vertical levels if MAXIMUM_DEPTH > 3000. The offending lines then look for an entry in the list of WOA09 vertical levels that exceeds the maximum depth of the domain. After 9250 m, the list ends but initialize_regridding keeps looking until it runs off the end of the list.

Workaround

Set DIAG_COORD_DEF_Z explicitly. If you don't need diagnostics on anything other than the native grid, DIAG_COORD_DEF_Z = "UNIFORM" works fine.

marshallward commented 1 year ago

I get a slightly different error (a MOM error, not a SIGSEGV), but otherwise the same result:

FATAL: MOM_diag_remap, initialize_regridding: For "WOA05:N" N must 0<N<41 but got "WOA09".

As you mentioned, the woa09_dz thicknesses end at 9250, so the lookup table for dz construction fails.

The segmentation fault has been "fixed", but this new error does not do a great job of directing the user to fix the problem, if all they want to do is run double gyre at a very large depth.

herrwang0 commented 1 year ago

Just to add a data point, I encountered the same error before with 1-layer barotropic runs. And I have also been using DIAG_COORD_DEF_Z = "UNIFORM" to fix the issue.

It is a bit counter-intuitive that subroutine initialize_regridding is called even with USE_REGRIDDING = False

pittwolfe commented 1 year ago

Agreed. It took me a while to track down the problem because I had USE_REGRIDDING = False and couldn't figure out why and when initialize_regridding was being called.

marshallward commented 1 year ago

According to @adcroft it's the diagnostics which need initialize_regridding, not the model fields. I agree that none of this feels very intuitive.

Hallberg-NOAA commented 5 months ago

This issue has been addressed and corrected by PR #636.