Datebug in MOM cap

[anton-seaice]

I configured 1deg_jra55do_iaf to produce daily restart files in nuopc.runconfig:

     restart_n = 1
     restart_option = ndays

I don't think any other changes I made are relevant, full details are here: https://github.com/COSIMA/MOM6-CICE6/commit/37efb5b5c446564be47d54962443297b9789d48d . Similarly the build i used is here: https://github.com/COSIMA/access-om3/compare/main...anton-seaice:access-om3:mom-date-bug-version

And then ran for 30ish days, on the model day starting 1958-01-29, MOM failed with the following error:

FATAL from PE    12: diag_time_inc: Invalid_date. Date=1958-02-29 00:00:00

Image              PC                Routine            Line        Source             
access-om3-MOM6-C  0000000004749D07  Unknown               Unknown  Unknown
access-om3-MOM6-C  000000000398AF17  mpp_mod_mp_mpp_er          71  mpp_util_mpi.inc
access-om3-MOM6-C  00000000037C2285  fms_mod_mp_fms_er         512  fms.F90
access-om3-MOM6-C  00000000036FFB23  diag_util_mod_mp_        1160  diag_util.F90
access-om3-MOM6-C  00000000036FEA1B  diag_util_mod_mp_        1080  diag_util.F90
access-om3-MOM6-C  00000000036C3B82  diag_manager_mod_         546  diag_manager.F90
access-om3-MOM6-C  00000000032C803F  mom_diag_manager_         213  MOM_diag_manager_infra.F90
access-om3-MOM6-C  0000000002ECE429  mom_diag_mediator        2524  MOM_diag_mediator.F90
access-om3-MOM6-C  0000000002ECD7F2  mom_diag_mediator        2378  MOM_diag_mediator.F90
access-om3-MOM6-C  0000000002ECBD4A  mom_diag_mediator        2120  MOM_diag_mediator.F90
access-om3-MOM6-C  0000000002ECB785  mom_diag_mediator         947  MOM_diag_mediator.F90
access-om3-MOM6-C  0000000002D13EF9  mom_mp_initialize        2902  MOM.F90
access-om3-MOM6-C  0000000002CC152A  mom_ocean_model_n         285  mom_ocean_model_nuopc.F90
access-om3-MOM6-C  0000000002C40C85  mom_cap_mod_mp_in         637  mom_cap.F90
access-om3-MOM6-C  0000000000D8A554  _ZN5ESMCI6FTable1        2167  ESMCI_FTable.C
access-om3-MOM6-C  0000000000D8E68A  ESMCI_FTableCallE         824  ESMCI_FTable.C
access-om3-MOM6-C  0000000001A7BD9F  _ZN5ESMCI3VMK5ent        2320  ESMCI_VMKernel.C
access-om3-MOM6-C  000000000216B2A2  _ZN5ESMCI2VM5ente        1216  ESMCI_VM.C
access-om3-MOM6-C  0000000000D8B997  c_esmc_ftablecall         981  ESMCI_FTable.C
access-om3-MOM6-C  0000000000BC274D  esmf_compmod_mp_e        1223  ESMF_Comp.F90
access-om3-MOM6-C  000000000124DC61  esmf_gridcompmod_        1412  ESMF_GridComp.F90
access-om3-MOM6-C  0000000000AD4B20  nuopc_driver_mp_l        2826  NUOPC_Driver.F90
access-om3-MOM6-C  0000000000A9D2E1  nuopc_driver_mp_i        1320  NUOPC_Driver.F90
access-om3-MOM6-C  0000000000D8A554  _ZN5ESMCI6FTable1        2167  ESMCI_FTable.C
access-om3-MOM6-C  0000000000D8E68A  ESMCI_FTableCallE         824  ESMCI_FTable.C
access-om3-MOM6-C  0000000001A7BD9F  _ZN5ESMCI3VMK5ent        2320  ESMCI_VMKernel.C
access-om3-MOM6-C  000000000216B2A2  _ZN5ESMCI2VM5ente        1216  ESMCI_VM.C
access-om3-MOM6-C  0000000000D8B997  c_esmc_ftablecall         981  ESMCI_FTable.C
access-om3-MOM6-C  0000000000BC274D  esmf_compmod_mp_e        1223  ESMF_Comp.F90
access-om3-MOM6-C  000000000124DC61  esmf_gridcompmod_        1412  ESMF_GridComp.F90
access-om3-MOM6-C  0000000000AD4B20  nuopc_driver_mp_l        2826  NUOPC_Driver.F90
access-om3-MOM6-C  0000000000A9D08E  nuopc_driver_mp_i        1315  NUOPC_Driver.F90
access-om3-MOM6-C  0000000000A82316  nuopc_driver_mp_i         476  NUOPC_Driver.F90
access-om3-MOM6-C  0000000000D8A554  _ZN5ESMCI6FTable1        2167  ESMCI_FTable.C
access-om3-MOM6-C  0000000000D8E68A  ESMCI_FTableCallE         824  ESMCI_FTable.C
access-om3-MOM6-C  0000000001A7BD9F  _ZN5ESMCI3VMK5ent        2320  ESMCI_VMKernel.C
access-om3-MOM6-C  000000000216B2A2  _ZN5ESMCI2VM5ente        1216  ESMCI_VM.C
access-om3-MOM6-C  0000000000D8B997  c_esmc_ftablecall         981  ESMCI_FTable.C
access-om3-MOM6-C  0000000000BC274D  esmf_compmod_mp_e        1223  ESMF_Comp.F90
access-om3-MOM6-C  000000000124DC61  esmf_gridcompmod_        1412  ESMF_GridComp.F90
access-om3-MOM6-C  0000000000430981  MAIN__                    128  esmApp.F90
access-om3-MOM6-C  0000000000430362  Unknown               Unknown  Unknown
libc-2.28.so       0000150D24C0ED85  __libc_start_main     Unknown  Unknown
access-om3-MOM6-C  000000000043026E  Unknown               Unknown  Unknown

1958-02-29 is indeed an invalid date, 1958 is not a leap year. However, it should be using 1958-01-29 as the date.

[anton-seaice]

Logs are in :

/scratch/tm70/as2285/experiments/archive/pio_blocksize

/scratch/tm70/as2285/experiments/work/pio_blocksize

[dougiesquire]

Can confirm that I can reproduce. Finally taking a look at this

[dougiesquire]

Though it's not very clear from the error message, this error comes about because the MOM output diagnostics are configured in a way that is incompatible with the run stop frequency:

• model runs in 1 day increments
• most MOM diagnostics output at 1 month frequency

Adjusting the MOM diag_table to output diagnostics at daily frequency fixes the issue:

 ### Section-1: File List
 #========================
-"GMOM_JRA.mom6.h.rho2%4yr-%2mo",                      1,  "months", 1, "days",   "time", 1, "months"
-"GMOM_JRA.mom6.h.native%4yr-%2mo",                    1,  "months", 1, "days",   "time", 1, "months"
-"GMOM_JRA.mom6.h.z%4yr-%2mo",                         1,  "months", 1, "days",   "time", 1, "months"
-"GMOM_JRA.mom6.h.sfc%4yr-%2mo",                       1,  "days",   1, "days",   "time", 1, "months"
+"GMOM_JRA.mom6.h.rho2%4yr-%2mo-%2dy",                 1,  "days",   1, "days",   "time", 1, "days"
+"GMOM_JRA.mom6.h.native%4yr-%2mo-%2dy",               1,  "days",   1, "days",   "time", 1, "days"
+"GMOM_JRA.mom6.h.z%4yr-%2mo-%2dy",                    1,  "days",   1, "days",   "time", 1, "days"
+"GMOM_JRA.mom6.h.sfc%4yr-%2mo-%2dy",                  1,  "days",   1, "days",   "time", 1, "days"
 "GMOM_JRA.mom6.h.static",                             -1, "days",   1, "days",   "time"