[BUG/ISSUE] geos-chem 13.0.0 has segmentation fault using intel 17 compiler on cray-xc40

[mirajnaik]

I am trying to run GEOS-CHEM 13.0.0 for 4x5 simulation on the cray-xc40 system with intel compiler but it is giving segmentation fault. . I have used default setting from createRunDir.sh script. but after exrtacting input .nc file is is giving segmentation fault

GEOS-CHEM version is updated. like 13.0.2 (how to check installed version)

I have tried with other simulation like 2*2.5 for intel and gnu compiler but issue is same. giving segmenation fault.

Log files

• Run logs (if applicable):

  
  ********************************************

• B e g i n T i m e S t e p p i n g !! *

  ---

---> DATE: 2019/07/01 UTC: 00:00 X-HRS: 0.000000 HEMCO already called for this timestep. Returning.

TPCORE_FVDAS (based on GMI) Tracer Transport Module successfully initialized

Application 173379742 exit codes: 174

• PBS error log: forrtl: severe (174): SIGSEGV, segmentation fault occurred forrtl: severe (174): SIGSEGV, segmentation fault occurred forrtl: severe (174): SIGSEGV, segmentation fault occurred forrtl: severe (174): SIGSEGV, segmentation fault occurred

• Run script (if applicable):# Env Variables:

MPIPROGINF='DETAIL' # program information about speed, vectorization export MPIPROGINF # {NO|YES|DETAIL} # MPISUSPEND='ON' # select waiting method export MPISUSPEND # {ON|OFF}={SUSPEND/RESUME|SPIN WAIT} # F_FTRACE='NO' # analysis list from compile option -ftrace export F_FTRACE # {NO|YES|FMT0|FMT1|FMT2} # F_ERRCNT=1 # stop execution after the first run time error export F_ERRCNT

#--- Start the run:

aprun -n 1 -N 1 ./gcclassic > GC.log

### Software versions
- CMake version: 1.17.4
- Compilers (Intel or GNU, and version): Intel 17.0.5.239
- NetCDF version: cray-netcdf-hdf5parallel/4.4.1.1

[yantosca]

Thanks for writing @mirajnaik. I think your issue is because you haven't maxed out the stacksize memory in your computational environment. See this section of our user manual: http://wiki.geos-chem.org/Specifying_settings_for_OpenMP_parallelization#OMP_STACKSIZE.

Long story short, temporary variables are created in a memory space called the stack. If you don't apply the settings to your Unix environment, then GEOS-Chem will think that there is less memory available than there really is.

[yantosca]

I believe this is a dupicate of issues #727 and #645.

[mirajnaik]

I have added STACK_SIZE in pbs script. but have still segmentation fault.

forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image              PC                Routine            Line        Source
gcclassic          0000000000D59D04  Unknown               Unknown  Unknown
libpthread-2.22.s  00002AAAAC7D1B20  Unknown               Unknown  Unknown
gcclassic          000000000044A807  drydep_mod_mp_upd         441  drydep_mod.F90
libiomp5.so        00002AAAACD90D43  __kmp_invoke_micr     Unknown  Unknown
libiomp5.so        00002AAAACD5F7AD  __kmp_fork_call       Unknown  Unknown
libiomp5.so        00002AAAACD3782E  __kmpc_fork_call      Unknown  Unknown
gcclassic          0000000000449CBB  drydep_mod_mp_upd         423  drydep_mod.F90
gcclassic          000000000044023F  drydep_mod_mp_do_         331  drydep_mod.F90
gcclassic          000000000040FED9  MAIN__                   1431  main.F90
gcclassic          000000000040F15E  Unknown               Unknown  Unknown
libc-2.22.so       00002AAAAD09F6D5  __libc_start_main     Unknown  Unknown
gcclassic          000000000040F069  Unknown               Unknown  Unknown

[yantosca]

@mirajnaik, have you looked at the line 441 in drydep_mod.F90? The error trace output that you show above points to this line as the source of the error. You could have a division by zero or an array index that is outside of array bounds.

I would recommend building the code from scratch with the debug flags on, i.e.

cd build
cmake ../CodeDir -DRUNDIR=.. -DCMAKE_BUILD_TYPE=Debug
make -j install
cd ..

then try to rerun the code and see if you get any extra error output.

[yantosca]

@mirajnaik, were you able to solve this issue?

[yantosca]

Going to close this issue due to a lack of activity. Feel free to open a new issue if you are still having problems.

[mirajnaik]

Dear yantosca,

I am trying to run the 2*2.5 simulation with the same setup. but getting a segmentation fault. can you suggest.