search

mom-ocean / MOM6

Modular Ocean Model
Other
184 stars 224 forks source link

new test case OM4_SIS_DT_THERM_7200 does not reproduce across a restart for longer runs #146

Closed nikizadehgfdl closed 4 years ago

nikizadehgfdl commented 9 years ago

1x2day = 2x1day but 1x10day != 2x5days

/// /lustre/f1/Niki.Zadeh/ulm_mom6_2015.04.20_0/OM4_SIS_DT_THERM_7200/ncrc2.intel-prod/1x0m2d_1152pe/restart/19000103.tar
\\\ /lustre/f1/Niki.Zadeh/ulm_mom6_2015.04.20_0/OM4_SIS_DT_THERM_7200/ncrc2.intel-prod/1x0m2d_2304pe/restart/19000103.tar
WARNING : NONESSENTIAL FILES DIFFER : input.nml
WARNING : NONESSENTIAL FILE NOT FOUND : OM4_SIS_DT_THERM_7200_1x0m2d_1152pe
/// /lustre/f1/Niki.Zadeh/ulm_mom6_2015.04.20_0/OM4_SIS_DT_THERM_7200/ncrc2.intel-prod/1x0m2d_1152pe/restart/19000103.tar
\\\ /lustre/f1/Niki.Zadeh/ulm_mom6_2015.04.20_0/OM4_SIS_DT_THERM_7200/ncrc2.intel-prod/2x0m1d_2304pe/restart/19000103.tar
WARNING : NONESSENTIAL FILES DIFFER : input.nml
WARNING : NONESSENTIAL FILE NOT FOUND : OM4_SIS_DT_THERM_7200_1x0m2d_1152pe
/// /lustre/f1/Niki.Zadeh/ulm_mom6_2015.04.20_0/OM4_SIS_DT_THERM_7200/ncrc2.intel-prod/1x0m10d_2997pe/restart/19000111.tar
\\\ /lustre/f1/Niki.Zadeh/ulm_mom6_2015.04.20_0/OM4_SIS_DT_THERM_7200/ncrc2.intel-prod/2x0m5d_2997pe/restart/19000111.tar
WARNING : NONESSENTIAL FILES DIFFER : input.nml
DIFFER : MOM.res_1.nc
DIFFER : MOM.res_3.nc
DIFFER : MOM.res.nc

Something must be breaking for runs longer than 2 days OR The non-reproducibility has something to do with the layout and mask_table.

I am going to try a 10day run with the layouts that reproduced.

nikizadehgfdl commented 9 years ago

This non-repro has something to do with the mask_table. The model reproduces across restarts when not using a mask_table !!

/// /lustre/f1/Niki.Zadeh/ulm_mom6_2015.04.24_0/OM4_SIS_DT_THERM_7200/ncrc2.intel-prod/1x0m10d_2304pe/restart/19000111.tar
\\\ /lustre/f1/Niki.Zadeh/ulm_mom6_2015.04.24_0/OM4_SIS_DT_THERM_7200/ncrc2.intel-prod/2x0m5d_2304pe/restart/19000111.tar
  CROSSOVER   PASSED: OM4_SIS_DT_THERM_7200
nikizadehgfdl commented 9 years ago

The non-repro is only in 3 restart variables h, h2 and age. They differ on a few isolated points on the map on the land !!

nikizadehgfdl commented 9 years ago

Given the interest in running with "spanning" option this needs to be resolved. It resembles the restart mismatch we have between pe-masked and non-pe-masked runs.

nikizadehgfdl commented 9 years ago

Odd enough this issue does not happen when I replace SIS1 with SIS2! The experiment OM4_SIS2_DT_THERM_7200 passes restart tests even with mask_table.

adcroft commented 9 years ago

!

Dr Alistair Adcroft (Alistair.Adcroft@noaa.gov) Princeton University Tel: (609) 987-5073 NOAA/GFDL, 201 Forrestal Road, Princeton, NJ 08540

On Wed, Aug 5, 2015 at 5:08 PM, Niki Zadeh notifications@github.com wrote:

Odd enough this issue does not happen when I replace SIS1 with SIS2! The experiment OM4_SIS2_DT_THERM_7200 passes restart tests even with mask_table.

— Reply to this email directly or view it on GitHub https://github.com/NOAA-GFDL/MOM6/issues/146#issuecomment-128148787.

Hallberg-NOAA commented 9 years ago

Perhaps this is yet another argument in favor of the widespread replacement of SIS with SIS2 in all new configurations... :smile:

Hallberg-NOAA commented 4 years ago

This issue may have been addressed in the years since it was first raised, but because SIS1 has been declared obsolete and is not in use for any active MOM6 development, and this issue never arose with SIS2, we have no way to verify whether it was, in fact corrected. In any case, it is no longer relevant, and is therefore appropriate to declare this issue closed. Please reopen this as a new issue if similar problems are ever encountered with MOM6/SIS2 configurations.