E3SM-Project / E3SM

Energy Exascale Earth System Model source code. NOTE: use "maint" branches for your work. Head of master is not validated.
https://docs.e3sm.org/E3SM
Other
339 stars 343 forks source link

Index '1' of dimension 1 of array 'tsec' above upper bound of 0 with `ERP_D_R_Ln10.ne4_ne4.FDPSCREAM-ARM97.pm-cpu_gnu` #6172

Open ndkeen opened 5 months ago

ndkeen commented 5 months ago

With master of Jan23, using ERP_D_R_Ln10.ne4_ne4.FDPSCREAM-ARM97.pm-cpu_gnu:

0: At line 569 of file /global/cfs/cdirs/e3sm/ndk/repos/me37-jan23/components/eam/src/control/iop_data_mod.F90
0: Fortran runtime error: Index '1' of dimension 1 of array 'tsec' above upper bound of 0
0:
0: Error termination. Backtrace:
0: #0  0x760639 in __iop_data_mod_MOD_setiopupdate
0:      at /global/cfs/cdirs/e3sm/ndk/repos/me37-jan23/components/eam/src/control/iop_data_mod.F90:569
0: #1  0x1191d64 in __stepon_MOD_stepon_run1
0:      at /global/cfs/cdirs/e3sm/ndk/repos/me37-jan23/components/eam/src/dynamics/se/stepon.F90:219
0: #2  0x653f17 in __cam_comp_MOD_cam_run1
0:      at /global/cfs/cdirs/e3sm/ndk/repos/me37-jan23/components/eam/src/control/cam_comp.F90:251
0: #3  0x643ed7 in __atm_comp_mct_MOD_atm_init_mct
0:      at /global/cfs/cdirs/e3sm/ndk/repos/me37-jan23/components/eam/src/cpl/atm_comp_mct.F90:523
0: #4  0x48dbc9 in __component_mod_MOD_component_init_cc
0:      at /global/cfs/cdirs/e3sm/ndk/repos/me37-jan23/driver-mct/main/component_mod.F90:248
0: #5  0x47e913 in __cime_comp_mod_MOD_cime_init
0:      at /global/cfs/cdirs/e3sm/ndk/repos/me37-jan23/driver-mct/main/cime_comp_mod.F90:2331
0: #6  0x48666c in cime_driver
0:      at /global/cfs/cdirs/e3sm/ndk/repos/me37-jan23/driver-mct/main/cime_driver.F90:122
0: #7  0x4867a8 in main

Also error with DEBUG intel

This test completed with next of March 1:

SMS_D_R_Ln10.ne4_ne4.FDPSCREAM-ARM97.pm-cpu_gnu
ndkeen commented 3 weeks ago

Well now (with master of June 17), I get a diff error for same test:

0: Program received signal SIGFPE: Floating-point exception - erroneous arithmetic operation.
0:
0: Backtrace for this error:
0: #0  0x148556053dbf in ???
0: #1  0xef670d in __radheat_MOD_radheat_tend
0:      at /global/cfs/cdirs/e3sm/ndk/repos/e3sm-jgfouca_cleanup_kokkos_tril/components/eam/src/physics/cam/radheat.F90:116
0: #2  0xabeab0 in __radiation_MOD_radiation_tend
0:      at /global/cfs/cdirs/e3sm/ndk/repos/e3sm-jgfouca_cleanup_kokkos_tril/components/eam/src/physics/rrtmgp/radiation.F90:1517
0: #3  0xe9dbc6 in tphysbc
0:      at /global/cfs/cdirs/e3sm/ndk/repos/e3sm-jgfouca_cleanup_kokkos_tril/components/eam/src/physics/cam/physpkg.F90:3057
0: #4  0xebfa11 in __physpkg_MOD_phys_run1._omp_fn.0
0:      at /global/cfs/cdirs/e3sm/ndk/repos/e3sm-jgfouca_cleanup_kokkos_tril/components/eam/src/physics/cam/physpkg.F90:1175
0: #5  0x14855b014295 in GOMP_parallel
0:      at ../../../cpe-gcc-11.2.0-202108140355.9bf1fd589a5c1/libgomp/parallel.c:178
0: #6  0xeb1f1a in __physpkg_MOD_phys_run1
0:      at /global/cfs/cdirs/e3sm/ndk/repos/e3sm-jgfouca_cleanup_kokkos_tril/components/eam/src/physics/cam/physpkg.F90:1154
0: #7  0x6659be in __cam_comp_MOD_cam_run1
0:      at /global/cfs/cdirs/e3sm/ndk/repos/e3sm-jgfouca_cleanup_kokkos_tril/components/eam/src/control/cam_comp.F90:268
0: #8  0x655405 in __atm_comp_mct_MOD_atm_init_mct
0:      at /global/cfs/cdirs/e3sm/ndk/repos/e3sm-jgfouca_cleanup_kokkos_tril/components/eam/src/cpl/atm_comp_mct.F90:523
0: #9  0x49d6ea in __component_mod_MOD_component_init_cc
0:      at /global/cfs/cdirs/e3sm/ndk/repos/e3sm-jgfouca_cleanup_kokkos_tril/driver-mct/main/component_mod.F90:248
0: #10  0x48e42a in __cime_comp_mod_MOD_cime_init
0:      at /global/cfs/cdirs/e3sm/ndk/repos/e3sm-jgfouca_cleanup_kokkos_tril/driver-mct/main/cime_comp_mod.F90:2373
0: #11  0x49618d in cime_driver
0:      at /global/cfs/cdirs/e3sm/ndk/repos/e3sm-jgfouca_cleanup_kokkos_tril/driver-mct/main/cime_driver.F90:122
0: #12  0x4962c9 in main
0:      at /global/cfs/cdirs/e3sm/ndk/repos/e3sm-jgfouca_cleanup_kokkos_tril/driver-mct/main/cime_driver.F90:23