Closed HomamBetar closed 5 years ago
Hi, Two comments before looking forward :
mpicc
and not mpiicpc
to compile C++
, it may not be a problem because it should have crash sooner. Moreover can you check and send the result of mpiicpc --version
or mpicc -version
?ls /opt/modulefiles/libraries/hdf5/1.10.1/intelmpi/lib
?
RegardsJulien
Hi,
mpiicpc --version icpc (ICC) 18.0.3 20180410 Copyright (C) 1985-2018 Intel Corporation. All rights reserved.
[bzu91@vm-zmc46 Smilei-4.1]$ mpiicc --version icc (ICC) 18.0.3 20180410 Copyright (C) 1985-2018 Intel Corporation. All rights reserved.
[bzu91@vm-zmc46 bzu91]$ ls /opt/modulefiles/libraries/hdf5/1.10.1/intelmpi/lib ls: impossible d'accéder à /opt/modulefiles/libraries/hdf5/1.10.1/intelmpi/lib: Aucun fichier ou dossier de ce type
It seems it does not find "lib" directory?
Now I am trying to compile with intel 2017. Thanks a lot for your help
It looks like your HDF5 directory is incorrect. There should be a lib
folder and an include
folder inside.
I will send an email to the cluster administrator. I compiled now the simulation with the following
1) intelmpi/2017.1.132 2) intel/2017.1.132 3) python/2.7/intel.2017-3 4) hdf5/1.10.1/intelmpi/intel17
I still have
Linking smilei ld: ne peut trouver -lhdf5
ll wait for the answer from cluster team, and then come back with the feedback.
Thank you very much
I have compiled the code with the following modules, and an executable smilei
has been generated.
1]- Currently Loaded Modules:
1) intel/2017.1.132 2) intelmpi/2017.1.132 3) python/2.7/intel.2017-3 4) hdf5/1.10.1/intelmpi/intel17
2]- I ran the simulation by submitting the follwoing submission file on the cluster (with two stream instability)
#!/bin/bash
#SBATCH -N 2
#SBATCH -p std
#SBATCH -J mpiSmilei
#SBATCH -n 64
#SBATCH --ntasks-per-node=32
#SBATCH -t 24:00:00
module purge
module load intel/2017.1.132 intelmpi/2017.1.132 python/2.7/intel.2017-3 hdf5/1.10.1/intelmpi/intel17
srun smilei < tst1d_02_two_str_instability.py
3]- The output file contain a segmentation fault as following:
more slurm-122042.out
srun: error: cna60: tasks 0-9,11-31: Segmentation fault
srun: Terminating job step 122042.0
slurmstepd: error: *** STEP 122042.0 ON cna60 CANCELLED AT 2019-05-31T15:54:22 ***
srun: error: cna62: tasks 32-63: Terminated
srun: error: cna60: task 10: Terminated
srun: Force Terminated job step 122042.0
I tried my best to figure out what is the problem, and until now I do not know.
2]- ldd smilei
gives
linux-vdso.so.1 => (0x00007ffde1eb3000)
libhdf5.so.101 => /opt/soft/hf/hdf5-1.10.1-intelmpi-intel/lib/libhdf5.so.101 (0x00002adfd4c4f000)
libpython2.7.so.1.0 => /opt/soft/all/intel/intelpython2/lib/libpython2.7.so.1.0 (0x00002adfd5316000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00002adfd5748000)
libdl.so.2 => /lib64/libdl.so.2 (0x00002adfd5965000)
libutil.so.1 => /lib64/libutil.so.1 (0x00002adfd5b69000)
libm.so.6 => /lib64/libm.so.6 (0x00002adfd5d6c000)
libmpicxx.so.12 => /opt/soft/all/intel/compilers_and_libraries_2017.1.132/linux/mpi/intel64/lib/libmpicxx.so.12 (0x00002adfd606f000)
libmpifort.so.12 => /opt/soft/all/intel/intelpython2/lib/libmpifort.so.12 (0x00002adfd628f000)
libmpi.so.12 => /opt/soft/all/intel/intelpython2/lib/libmpi.so.12 (0x00002adfd664e000)
librt.so.1 => /lib64/librt.so.1 (0x00002adfd737d000)
libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00002adfd7585000)
libiomp5.so => /opt/soft/all/intel/intelpython2/lib/libiomp5.so (0x00002adfd788d000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002adfd7c30000)
libc.so.6 => /lib64/libc.so.6 (0x00002adfd7e46000)
libz.so.1 => /opt/soft/all/intel/intelpython2/lib/libz.so.1 (0x00002adfd8209000)
libimf.so => /opt/soft/all/intel/intelpython2/lib/libimf.so (0x00002adfd8425000)
libsvml.so => /opt/soft/all/intel/intelpython2/lib/libsvml.so (0x00002adfd8923000)
libirng.so => /opt/soft/all/intel/intelpython2/lib/libirng.so (0x00002adfd982f000)
libintlc.so.5 => /opt/soft/all/intel/intelpython2/lib/libintlc.so.5 (0x00002adfd9ba2000)
/lib64/ld-linux-x86-64.so.2 (0x000055e9e554e000)
3]- make env
gives
SMILEICXX : mpiicpc
PYTHONEXE : python
MPIVERSION :
VERSION : ??-??
OPENMP_FLAG : -fopenmp -D_OMP
HDF5_ROOT_DIR : /opt/soft/std/hdf5-1.10.1-intelmpi-intel
SITEDIR : /home/zmc46/bzu91/.local/lib/python2.7/site-packages
PY_CXXFLAGS : -I/opt/soft/all/intel/intelpython2/include/python2.7 -I/opt/soft/all/intel/intelpython2/include/python2.7 -I/opt/soft/all/intel/intelpython2/lib/python2.7/site-packages/numpy/core/include -DSMILEI_USE_NUMPY -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION
PY_LDFLAGS : -lpython2.7 -lpthread -ldl -lutil -lm -Xlinker -export-dynamic
CXXFLAGS : -D__VERSION=\"??-??\" -D_VECTO -std=c++11 -Wall -I/opt/soft/std/hdf5-1.10.1-intelmpi-intel/include -Isrc -Isrc/Checkpoint -Isrc/Collisions -Isrc/Diagnostic -Isrc/DomainDecomposition -Isrc/ElectroMagn -Isrc/ElectroMagnBC -Isrc/ElectroMagnSolver -Isrc/Field -Isrc/Interpolator -Isrc/Ionization -Isrc/MovWindow -Isrc/MultiphotonBreitWheeler -Isrc/Params -Isrc/Patch -Isrc/picsar_interface -Isrc/Profiles -Isrc/Projector -Isrc/Pusher -Isrc/Python -Isrc/Radiation -Isrc/SmileiMPI -Isrc/Species -Isrc/Tools -Ibuild/src/Python -I/opt/soft/all/intel/intelpython2/include/python2.7 -I/opt/soft/all/intel/intelpython2/include/python2.7 -I/opt/soft/all/intel/intelpython2/lib/python2.7/site-packages/numpy/core/include -DSMILEI_USE_NUMPY -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -D_VECTO -O3 -g -fopenmp -D_OMP
LDFLAGS : -L/opt/soft/std/hdf5-1.10.1-intelmpi-intel/lib -lhdf5 -lpython2.7 -lpthread -ldl -lutil -lm -Xlinker -export-dynamic -lm -fopenmp -D_OMP
4]- make config=verbose
gives
Unknown smilei version
make: Rien à faire pour « default ».
5]- echo $LD_LIBRARY_PATH
gives
/opt/soft/hf/hdf5-1.10.1-intelmpi-intel/lib:/opt/soft/all/intel/intelpython2/lib:/opt/soft/all/intel/compilers_and_libraries_2017.1.132/linux/mpi/intel64/lib:/opt/soft/all/intel/compilers_and_libraries_2017.1.132/linux/mpi/mic/lib:/opt/soft/all/intel/compilers_and_libraries_2017.1.132/linux/compiler/lib/intel64:/opt/soft/all/intel/compilers_and_libraries_2017.1.132/linux/compiler/lib/intel64_lin:/opt/soft/all/intel/compilers_and_libraries_2017.1.132/linux/ipp/lib/intel64:/opt/soft/all/intel/compilers_and_libraries_2017.1.132/linux/compiler/lib/intel64_lin:/opt/soft/all/intel/compilers_and_libraries_2017.1.132/linux/tbb/lib/intel64/gcc4.7:/opt/soft/all/intel/debugger_2017/iga/lib:/opt/soft/all/intel/debugger_2017/libipt/intel64/lib:/opt/soft/all/intel/compilers_and_libraries_2017.1.132/linux/daal/lib/intel64_lin:/opt/soft/all/intel/compilers_and_libraries_2017.1.132/linux/tbb/lib/intel64_lin/gcc4.4
@jderouillat @mccoys any help?
I really appreciate your help as I'm not an expert with clusters.
Thanks a lot
Dear Homam,
This input file (tst1d_02_two_str_instabilitity) uses 16 patches. This means that you can use up to 16 cores to run it. It seems to me that you try to use way to much resources for this (light) simulation. Can you have a try using eg. 2MPI et 8openMP threads?
Cheers, M
On Fri, May 31, 2019 at 4:25 PM HomamBetar notifications@github.com wrote:
I have compiled the code with the following modules, and an executable smilei has been generated. 1]- Currently Loaded Modules:
1) intel/2017.1.132 2) intelmpi/2017.1.132 3) python/2.7/intel.2017-3 4) hdf5/1.10.1/intelmpi/intel17
2]- I ran the simulation by submitting the follwoing submission file on the cluster (with two stream instability)
!/bin/bash #SBATCH -N 2 #SBATCH -p std #SBATCH -J mpiSmilei #SBATCH -n 64
SBATCH --ntasks-per-node=32 #SBATCH -t 24:00:00 module purge module load
intel/2017.1.132 intelmpi/2017.1.132 python/2.7/intel.2017-3 hdf5/1.10.1/intelmpi/intel17 srun smilei < tst1d_02_two_str_instability.py
3]- The output file contain a segmentation fault as following: more slurm-122042.out srun: error: cna60: tasks 0-9,11-31: Segmentation fault srun: Terminating job step 122042.0 slurmstepd: error: STEP 122042.0 ON cna60 CANCELLED AT 2019-05-31T15:54:22 srun: error: cna62: tasks 32-63: Terminated srun: error: cna60: task 10: Terminated srun: Force Terminated job step 122042.0
I tried my best to figure out what is the problem, and until now I do not know.
2]- ldd smilei gives linux-vdso.so.1 => (0x00007ffde1eb3000) libhdf5.so.101 => /opt/soft/hf/hdf5-1.10.1-intelmpi-intel/lib/libhdf5.so.101 (0x00002adfd4c4f000) libpython2.7.so.1.0 => /opt/soft/all/intel/intelpython2/lib/libpython2.7.so.1.0 (0x00002adfd5316000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00002adfd5748000) libdl.so.2 => /lib64/libdl.so.2 (0x00002adfd5965000) libutil.so.1 => /lib64/libutil.so.1 (0x00002adfd5b69000) libm.so.6 => /lib64/libm.so.6 (0x00002adfd5d6c000) libmpicxx.so.12 => /opt/soft/all/intel/compilers_and_libraries_2017.1.132/linux/mpi/intel64/lib/libmpicxx.so.12 (0x00002adfd606f000) libmpifort.so.12 => /opt/soft/all/intel/intelpython2/lib/libmpifort.so.12 (0x00002adfd628f000) libmpi.so.12 => /opt/soft/all/intel/intelpython2/lib/libmpi.so.12 (0x00002adfd664e000) librt.so.1 => /lib64/librt.so.1 (0x00002adfd737d000) libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00002adfd7585000) libiomp5.so => /opt/soft/all/intel/intelpython2/lib/libiomp5.so (0x00002adfd788d000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002adfd7c30000) libc.so.6 => /lib64/libc.so.6 (0x00002adfd7e46000) libz.so.1 => /opt/soft/all/intel/intelpython2/lib/libz.so.1 (0x00002adfd8209000) libimf.so => /opt/soft/all/intel/intelpython2/lib/libimf.so (0x00002adfd8425000) libsvml.so => /opt/soft/all/intel/intelpython2/lib/libsvml.so (0x00002adfd8923000) libirng.so => /opt/soft/all/intel/intelpython2/lib/libirng.so (0x00002adfd982f000) libintlc.so.5 => /opt/soft/all/intel/intelpython2/lib/libintlc.so.5 (0x00002adfd9ba2000) /lib64/ld-linux-x86-64.so.2 (0x000055e9e554e000)
3]- make env gives SMILEICXX : mpiicpc PYTHONEXE : python MPIVERSION : VERSION : ??-?? OPENMP_FLAG : -fopenmp -D_OMP HDF5_ROOT_DIR : /opt/soft/std/hdf5-1.10.1-intelmpi-intel SITEDIR : /home/zmc46/bzu91/.local/lib/python2.7/site-packages PY_CXXFLAGS : -I/opt/soft/all/intel/intelpython2/include/python2.7 -I/opt/soft/all/intel/intelpython2/include/python2.7 -I/opt/soft/all/intel/intelpython2/lib/python2.7/site-packages/numpy/core/include -DSMILEI_USE_NUMPY -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION PY_LDFLAGS : -lpython2.7 -lpthread -ldl -lutil -lm -Xlinker -export-dynamic CXXFLAGS : -D__VERSION=\"??-??\" -D_VECTO -std=c++11 -Wall -I/opt/soft/std/hdf5-1.10.1-intelmpi-intel/include -Isrc -Isrc/Checkpoint -Isrc/Collisions -Isrc/Diagnostic -Isrc/DomainDecomposition -Isrc/ElectroMagn -Isrc/ElectroMagnBC -Isrc/ElectroMagnSolver -Isrc/Field -Isrc/Interpolator -Isrc/Ionization -Isrc/MovWindow -Isrc/MultiphotonBreitWheeler -Isrc/Params -Isrc/Patch -Isrc/picsar_interface -Isrc/Profiles -Isrc/Projector -Isrc/Pusher -Isrc/Python -Isrc/Radiation -Isrc/SmileiMPI -Isrc/Species -Isrc/Tools -Ibuild/src/Python -I/opt/soft/all/intel/intelpython2/include/python2.7 -I/opt/soft/all/intel/intelpython2/include/python2.7 -I/opt/soft/all/intel/intelpython2/lib/python2.7/site-packages/numpy/core/include -DSMILEI_USE_NUMPY -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -D_VECTO -O3 -g -fopenmp -D_OMP LDFLAGS : -L/opt/soft/std/hdf5-1.10.1-intelmpi-intel/lib -lhdf5 -lpython2.7 -lpthread -ldl -lutil -lm -Xlinker -export-dynamic -lm -fopenmp -D_OMP
4]- make config=verbose gives
Unknown smilei version make: Rien à faire pour « default ».
5]- echo $LD_LIBRARY_PATH gives
/opt/soft/hf/hdf5-1.10.1-intelmpi-intel/lib:/opt/soft/all/intel/intelpython2/lib:/opt/soft/all/intel/compilers_and_libraries_2017.1.132/linux/mpi/intel64/lib:/opt/soft/all/intel/compilers_and_libraries_2017.1.132/linux/mpi/mic/lib:/opt/soft/all/intel/compilers_and_libraries_2017.1.132/linux/compiler/lib/intel64:/opt/soft/all/intel/compilers_and_libraries_2017.1.132/linux/compiler/lib/intel64_lin:/opt/soft/all/intel/compilers_and_libraries_2017.1.132/linux/ipp/lib/intel64:/opt/soft/all/intel/compilers_and_libraries_2017.1.132/linux/compiler/lib/intel64_lin:/opt/soft/all/intel/compilers_and_libraries_2017.1.132/linux/tbb/lib/intel64/gcc4.7:/opt/soft/all/intel/debugger_2017/iga/lib:/opt/soft/all/intel/debugger_2017/libipt/intel64/lib:/opt/soft/all/intel/compilers_and_libraries_2017.1.132/linux/daal/lib/intel64_lin:/opt/soft/all/intel/compilers_and_libraries_2017.1.132/linux/tbb/lib/intel64_lin/gcc4.4
@jderouillat https://github.com/jderouillat @mccoys https://github.com/mccoys any help? I really appreciate your help as I'm not an expert with clusters. Thanks a lot
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/SmileiPIC/Smilei/issues/116?email_source=notifications&email_token=AENR6TBRZZJFWUO5YNGD6G3PYEYNFA5CNFSM4HP32BYKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWVLRYI#issuecomment-497727713, or mute the thread https://github.com/notifications/unsubscribe-auth/AENR6TAJA2NXUJMZFC6YAWLPYEYNFANCNFSM4HP32BYA .
Also, please indicate at what point the simulation crash. For example, submit the full output log.
And you could also try smilei_test in your login node (not submitting a job) to see if this works.
Just type smilei_test path/to/simulation.py
I tried smilei_test
, but always segfault
[bzu91@vm-zmc46 Smilei-4.1]$ ls benchmarks happi scripts slurm-122407.out src validation build License slurm-122042.out smilei subSmilei.slurm databases makefile slurm-122404.out smilei.sh tst1d_02_two_str_instability.py doc README.md slurm-122405.out smilei_test tst1d_07_cir_plane_wave_rela.py
[bzu91@vm-zmc46 Smilei-4.1]$ ./smilei_test tst1d_02_two_str_instability.py
Erreur de segmentation
[bzu91@vm-zmc46 Smilei-4.1]$
output file is attached to this comment. [slurm-122042.txt](https://github.com/SmileiPIC/Smilei/files/3247825/slurm-122042.txt
The program does not even start it seems. There must be something wrong in your setup.
First things first: have you cleaned your build after you made changes in your environment?
make clean
and make
again.
@mccoys @MickaelGrechX @jderouillat Thanks a lot for your help. Everything is working fine now and the simulation has performed successfully. Should I submit submit the submission file on EXPLOR cluster here? or send it by email?
As you wish. You can propose a pull request with your new file, or send it to one of us.
Dears,
I have tried to run the simulation (Smilei-4.1) on EXPLOR cluster . With first
1) intelmpi/2018.0.128 2) python/2.7/intel.2017-3 3) hdf5/1.10.1/intelmpi/intel17 4) intel/2018.3
In makefile file, I made the following modificationsthe main problems were
problem in openmp work, such as
Linking smilei /usr/bin/ld: ne peut trouver -lhdf5 collect2: erreur: ld a retourné 1 code d'état d'exécution
results of
make env
Results of
make config=verbose
The Resutls of the Compiling
Thanks very much Cheers