search

SmileiPIC / Smilei

Particle-in-cell code for plasma simulation
https://smileipic.github.io/Smilei
333 stars 119 forks source link

Compiling on host problem #84

Closed emil-mouz closed 5 years ago

emil-mouz commented 5 years ago

I'm trying to compile SMILEI in a host server and there is this problem pop-up. Anyone know what may be the case? screenshot from 2019-02-08 15-44-48

iltommi commented 5 years ago

1) could you paste the command make config=verbose ? 2) could you paste the command make env ? 3) what system are you compiling Smilei ? 4) what is your mpic++ --version and mpirun --version ? 5) can you cimpile and run a simple program like this? https://github.com/wesleykendall/mpitutorial/blob/gh-pages/tutorials/mpi-hello-world/code/mpi_hello_world.c 6) if the last point didn't work, are you sure the MPI installation was properly done? Smilei doesn't use the libssl but it's probably required by MPI.

mccoys commented 5 years ago

@emil-mouz did you manage to compile or run @iltommi commands ?

emil-mouz commented 5 years ago

Not yet, I will try again to compile SMILEI in cluster maybe tomorrow and I will post the results asap. Thanks for asking.

Στις Τετάρτη, 13 Φεβρουαρίου 2019, ο χρήστης mccoys < notifications@github.com> έγραψε:

@emil-mouz https://github.com/emil-mouz did you manage to compile or run @iltommi https://github.com/iltommi commands ?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/SmileiPIC/Smilei/issues/84#issuecomment-463293581, or mute the thread https://github.com/notifications/unsubscribe-auth/ArnwqFzyrmnroKVdTeeBceOqbxL7OqOaks5vNE1ugaJpZM4avvqQ .

emil-mouz commented 5 years ago

First of all thank you for your replies and your interest so far.

Well, my problem is, and it's not SMILEI's problem, that i try to load some modules for the SMILEI in a host server so I can compile it. I know for sure that SMILEI requires hdf5, openmpi and python in order to run. So, I've tried many differents version of openmpi, eg 1.8.2-intel, gcc and newer ones and many different python libraries like 2.7 and 3.6, but I cannot "find" the right combination or I don't know what's the problem is since different errors pop up.
Also, in the SMILEI installation page you doesn't mention which openmpi or hdf5 version works? SMILEI works for every version of openmpi, hdf5 and python or it should have been a specific one? Also, I cannot totally understand why and which version of openmpi and python should I choose between intel and gcc. I know that they are novice questions but I would be happy if you could clarify some basics things for me that for sure they will be helpful for me. Thank you for you time.

mccoys commented 5 years ago

Hdf5 versions 1.8 and 1.10 should work. Python 2.7 and above should work.

Openmpi 1.8 and 2.0 work with gcc. Not sure about v3.

Intel Mpi 2016+ should work.

Now, the trick is to have all of these compiled with the same environment. Additionally, we recommend to get MPI that supports MPI_THREAD_MULTIPLE which is not the default. If you don't have this, smilei must be compiled with make config=no_mpi_tm.

Have you tried the tests suggested by @iltommi? We could help you maybe.

Otherwise you must ask for help with your system admin because it is hard for us to help without access to the machine. If it is a large machine, maybe we have already used it? If you can, tell us which one it is

emil-mouz commented 5 years ago

In trying to compile SMILEI in metacentrum which has like 7-8 host servers and different clusters around czech republic. If I'm not mistaken every host server has the same modules available.

I load the following modules and different version of them: openmpi-2.1.1 gcc, hdf5 1.10. 0 gcc, python 2.7.10 gcc and gcc 4.9

emil-mouz commented 5 years ago

screenshot from 2019-02-19 02-12-28

emil-mouz commented 5 years ago

screenshot from 2019-02-19 02-21-34

Different error with different modules

jderouillat commented 5 years ago

Regarding your 2 screenshots, I suggest you to use the environment of the first where it's the link edition which fails, an adjustment regarding Python seems to be necessary.
As suggested by @iltommi , can you provide the result of make config=verbose. It may give us informations of what is missing.

emil-mouz commented 5 years ago 
[emil_mouz_91@zuphux Smilei-4.1]$ make config=verbose
Unknown smilei version
Linking smilei
mpicxx build/src/Checkpoint/Checkpoint.o build/src/Collisions/CollisionalIonization.o build/src/Collisions/Collisions.o build/src/Collisions/CollisionsSingle.o build/src/Diagnostic/DiagnosticPerformances.o build/src/Diagnostic/DiagnosticScalar.o build/src/Diagnostic/DiagnosticParticleBinning.o build/src/Diagnostic/Histogram.o build/src/Diagnostic/DiagnosticFieldsAM.o build/src/Diagnostic/DiagnosticCartFields.o build/src/Diagnostic/TimeSelection.o build/src/Diagnostic/DiagnosticFields3D.o build/src/Diagnostic/DiagnosticCartFields2D.o build/src/Diagnostic/DiagnosticFields2D.o build/src/Diagnostic/DiagnosticCartFields3D.o build/src/Diagnostic/DiagnosticProbes.o build/src/Diagnostic/DiagnosticFields.o build/src/Diagnostic/DiagnosticFields1D.o build/src/Diagnostic/DiagnosticScreen.o build/src/Diagnostic/DiagnosticTrack.o build/src/DomainDecomposition/HilbertDomainDecomposition.o build/src/DomainDecomposition/Hilbert_functions.o build/src/DomainDecomposition/GlobalDomainDecomposition.o build/src/DomainDecomposition/CartesianDomainDecomposition.o build/src/ElectroMagn/ElectroMagn1D.o build/src/ElectroMagn/Laser.o build/src/ElectroMagn/ElectroMagn.o build/src/ElectroMagn/LaserPropagator.o build/src/ElectroMagn/ElectroMagn2D.o build/src/ElectroMagn/ElectroMagnAM.o build/src/ElectroMagn/LaserEnvelope.o build/src/ElectroMagn/ElectroMagn3D.o build/src/ElectroMagnBC/ElectroMagnBC.o build/src/ElectroMagnBC/ElectroMagnBC3D_SM.o build/src/ElectroMagnBC/ElectroMagnBC1D_refl.o build/src/ElectroMagnBC/ElectroMagnBC1D_SM.o build/src/ElectroMagnBC/ElectroMagnBC1D.o build/src/ElectroMagnBC/ElectroMagnBC3D.o build/src/ElectroMagnBC/ElectroMagnBCAM_Axis.o build/src/ElectroMagnBC/ElectroMagnBC2D_Trans_Damping.o build/src/ElectroMagnBC/ElectroMagnBC2D.o build/src/ElectroMagnBC/EnvelopeBC3D_refl.o build/src/ElectroMagnBC/ElectroMagnBC3D_refl.o build/src/ElectroMagnBC/ElectroMagnBC2D_SM.o build/src/ElectroMagnBC/EnvelopeBC.o build/src/ElectroMagnBC/ElectroMagnBC3D_BM.o build/src/ElectroMagnBC/ElectroMagnBC2D_refl.o build/src/ElectroMagnBC/ElectroMagnBCAM_SM.o build/src/ElectroMagnBC/ElectroMagnBCAM_BM.o build/src/ElectroMagnSolver/MF_Solver2D_Lehe.o build/src/ElectroMagnSolver/MA_SolverAM_norm.o build/src/ElectroMagnSolver/MF_Solver3D_Yee.o build/src/ElectroMagnSolver/MA_Solver3D_norm.o build/src/ElectroMagnSolver/MF_Solver3D_Lehe.o build/src/ElectroMagnSolver/MF_Solver2D_GrassiSpL.o build/src/ElectroMagnSolver/PXR_Solver3D_FDTD.o build/src/ElectroMagnSolver/MF_Solver2D_Grassi.o build/src/ElectroMagnSolver/MF_SolverAM_Yee.o build/src/ElectroMagnSolver/PXR_Solver3D_GPSTD.o build/src/ElectroMagnSolver/MF_Solver1D_Yee.o build/src/ElectroMagnSolver/PXR_Solver2D_GPSTD.o build/src/ElectroMagnSolver/MF_Solver2D_Yee.o build/src/ElectroMagnSolver/MA_Solver2D_Friedman.o build/src/ElectroMagnSolver/MF_Solver2D_Cowan.o build/src/ElectroMagnSolver/MA_Solver1D_norm.o build/src/ElectroMagnSolver/MA_Solver2D_norm.o build/src/Field/Field1D.o build/src/Field/Field3D.o build/src/Field/Field2D.o build/src/Field/cField3D.o build/src/Field/cField2D.o build/src/Interpolator/Interpolator3D2OrderV.o build/src/Interpolator/Interpolator3D.o build/src/Interpolator/InterpolatorAM2Order.o build/src/Interpolator/Interpolator2D2Order.o build/src/Interpolator/Interpolator3D2Order.o build/src/Interpolator/Interpolator1D.o build/src/Interpolator/Interpolator2D4Order.o build/src/Interpolator/Interpolator1D3Order.o build/src/Interpolator/Interpolator1D2Order.o build/src/Interpolator/Interpolator2D2OrderV.o build/src/Interpolator/InterpolatorAM.o build/src/Interpolator/Interpolator.o build/src/Interpolator/Interpolator3D4OrderV.o build/src/Interpolator/Interpolator1D4Order.o build/src/Interpolator/Interpolator3D4Order.o build/src/Interpolator/Interpolator2D.o build/src/Ionization/IonizationFromRate.o build/src/Ionization/Ionization.o build/src/Ionization/IonizationTables.o build/src/Ionization/IonizationTunnel.o build/src/MovWindow/SimWindow.o build/src/MultiphotonBreitWheeler/MultiphotonBreitWheeler.o build/src/MultiphotonBreitWheeler/MultiphotonBreitWheelerTables.o build/src/Params/Params.o build/src/Params/PeekAtSpecies.o build/src/Params/OpenPMDparams.o build/src/Patch/Patch.o build/src/Patch/PatchAM.o build/src/Patch/SyncVectorPatch.o build/src/Patch/Domain.o build/src/Patch/Patch2D.o build/src/Patch/Patch3D.o build/src/Patch/SyncCartesianPatch.o build/src/Patch/Patch1D.o build/src/Patch/VectorPatch.o build/src/picsar_interface/interface.o build/src/Profiles/Profile.o build/src/Profiles/Function.o build/src/Projector/Projector3D.o build/src/Projector/ProjectorAM.o build/src/Projector/Projector3D4OrderV.o build/src/Projector/Projector1D4Order.o build/src/Projector/ProjectorAM2Order.o build/src/Projector/Projector.o build/src/Projector/Projector2D4Order.o build/src/Projector/Projector1D2Order.o build/src/Projector/Projector1D.o build/src/Projector/Projector3D4Order.o build/src/Projector/Projector2D2OrderV.o build/src/Projector/Projector3D2OrderV.o build/src/Projector/Projector2D.o build/src/Projector/Projector3D2Order.o build/src/Projector/Projector2D2Order.o build/src/Pusher/PusherVay.o build/src/Pusher/PusherPonderomotiveBoris.o build/src/Pusher/PusherBoris.o build/src/Pusher/Pusher.o build/src/Pusher/PusherPhoton.o build/src/Pusher/PusherHigueraCary.o build/src/Pusher/PusherPonderomotiveBorisV.o build/src/Pusher/PusherRRLL.o build/src/Pusher/PusherBorisNR.o build/src/Pusher/PusherPonderomotivePositionBorisV.o build/src/Pusher/PusherPonderomotivePositionBoris.o build/src/Pusher/PusherBorisV.o build/src/Radiation/RadiationLandauLifshitz.o build/src/Radiation/RadiationNiel.o build/src/Radiation/RadiationCorrLandauLifshitz.o build/src/Radiation/Radiation.o build/src/Radiation/RadiationMonteCarlo.o build/src/Radiation/RadiationTables.o build/src/Smilei.o build/src/SmileiMPI/SmileiMPI_test.o build/src/SmileiMPI/SmileiMPI.o build/src/SmileiMPI/AsyncMPIbuffers.o build/src/Species/SpeciesNorm.o build/src/Species/Particles.o build/src/Species/PartWall.o build/src/Species/SpeciesV.o build/src/Species/PartBoundCond.o build/src/Species/Particle.o build/src/Species/SpeciesNormV.o build/src/Species/SpeciesAdaptiveV.o build/src/Species/SpeciesMetrics.o build/src/Species/SpeciesAdaptiveV2.o build/src/Species/Species.o build/src/Tools/tabulatedFunctions.o build/src/Tools/backward.o build/src/Tools/Timer.o build/src/Tools/Timers.o build/src/Tools/userFunctions.o build/src/Tools/Tools.o -o build/smilei -lhdf5 -lpython2.7 -lpthread -ldl -lutil -lm -Xlinker -export-dynamic -lm -fopenmp -D_OMP
/usr/bin/ld: cannot find -lpython2.7
collect2: error: ld returned 1 exit status
make: *** [smilei] Error 1
iltommi commented 5 years ago

and what about make env (point 2 of my previous comment: https://github.com/SmileiPIC/Smilei/issues/84#issuecomment-461809646) ?

emil-mouz commented 5 years ago 
[emil_mouz_91@zuphux Smilei-4.1]$ make env
Unknown smilei version
SMILEICXX : mpicxx
PYTHONEXE : python
MPIVERSION : 
VERSION : ??-??
OPENMP_FLAG : -fopenmp -D_OMP
HDF5_ROOT_DIR : 
SITEDIR : /storage/brno3-cerit/home/emil_mouz_91/.local/lib/python2.7/site-packages
PY_CXXFLAGS : -I/software/python/2.7.10/gcc/include/python2.7 -I/software/python/2.7.10/gcc/include/python2.7
PY_LDFLAGS : -lpython2.7 -lpthread -ldl -lutil -lm -Xlinker -export-dynamic
CXXFLAGS : -D__VERSION=\"??-??\" -D_VECTO -std=c++11 -Wall -Isrc -Isrc/Params -Isrc/ElectroMagnSolver -Isrc/ElectroMagn -Isrc/ElectroMagnBC -Isrc/Radiation -Isrc/Ionization -Isrc/Interpolator -Isrc/Collisions -Isrc/Tools -Isrc/Python -Isrc/Projector -Isrc/DomainDecomposition -Isrc/MovWindow -Isrc/Profiles -Isrc/picsar_interface -Isrc/Checkpoint -Isrc/Pusher -Isrc/Field -Isrc/MultiphotonBreitWheeler -Isrc/SmileiMPI -Isrc/Species -Isrc/Diagnostic -Isrc/Patch -Ibuild/src/Python -I/software/python/2.7.10/gcc/include/python2.7 -I/software/python/2.7.10/gcc/include/python2.7 -D_VECTO -O3 -g  -fopenmp -D_OMP
LDFLAGS : -lhdf5 -lpython2.7 -lpthread -ldl -lutil -lm -Xlinker -export-dynamic -lm -fopenmp -D_OMP
iltommi commented 5 years ago

it looks like python library is not in the default path and smilei can;t figure out where it is located

could you please give us the output of echo $LD_LIBRARY_PATH

and

python -c 'from distutils import sysconfig; print sysconfig.get_config_var("LIBDIR")'

emil-mouz commented 5 years ago

I was thinking the same thing to tell you the truth...

[emil_mouz_91@zuphux Smilei-4.1]$ echo $LD_LIBRARY_PATH
/afs/ics.muni.cz/software/gcc-4.8.1/lib:/afs/ics.muni.cz/software/gcc-4.8.1/lib64:/software/mpc-0.8.2/lib:/software/mpfr-2.4.1/lib:/software/gmp-4.2.4/lib:/software/python/2.7.10/gcc/lib:/software/hdf5/1.10.0/gcc/lib:/software/openmpi/2.1.1/gcc/lib:/usr/local/lib64

[emil_mouz_91@zuphux Smilei-4.1]$ python -c 'from distutils import sysconfig; print sysconfig.get_config_var("LIBDIR")'
/software/python/2.7.10/gcc/lib
jderouillat commented 5 years ago

Can you set the environment variable PYTHONHOME to the directory in which there is lib/libpython2.7.so ?
Maybe : 

export PYTHONHOME=/software/python/2.7.10/gcc/
jderouillat commented 5 years ago

You'll have to set HDF5_ROOT_DIR which is not set for now. 

export HDF5_ROOT_DIR=/software/hdf5/1.10.0/gcc
emil-mouz commented 5 years ago

It's compiled just with the command export PYTHONHOME !

I will try to run a test case now to see if everything works fine

emil-mouz commented 5 years ago

Well, I tried to for example to run the following bash file in the cluster Smilei_test.txt

And i got the following errors, and outputs. (The #PBS commands are fine I've tried them with different codes)

Test6.errors_results.txt Test6.output_results.txt

jderouillat commented 5 years ago

Looking at your batch script in Smilei_test.txt, it appears that you don't copy your binary in your working directory First_test : 

...
make clean
make -j 10
cd First_test
date
mpirun -n 10 ./smilei First_simulation.py
...
emil-mouz commented 5 years ago

I cp the binary and at least SMILEI seems to start running. I also add numpy-1.11.1-py2.7.10

Smilei_run_2.txt

But, I got the following different errors.

Test7.output.txt Test7.errors.txt

jderouillat commented 5 years ago

A numpy dependency is missing in your batch environment, you should load lapack :

[Python] ImportError: /software/numpy/1.11.1/lib/python2.7/site-packages/numpy-1.11.1-py2.7-linux-x86_64.egg/numpy/core/multiarray.so: undefined symbol: cblas_cdotc_sub
emil-mouz commented 5 years ago

It seems that Metacetrum doesn't have the lapack module available so I cannot load it. Any alternatives? Should I try different version of python or try run it without numpy?

emil-mouz commented 5 years ago

Ok, at last I remove numpy from input file and it runs.!

mccoys commented 5 years ago

Numpy is not required but can help in some cases. If you ever need it, you can make a fresh installation in your home directory. Usually, the installation is made in ~/.local/python/ or something like that.

emil-mouz commented 5 years ago

What about happi module and pint module? The same applies for them?

Στις Πέμ, 21 Φεβ 2019 στις 1:04 μ.μ., ο/η mccoys notifications@github.com έγραψε:

Numpy is not required but can help in some cases. If you ever need it, you can make a fresh installation in your home directory. Usually, the installation is made in ~/.local/python/ or something like that.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/SmileiPIC/Smilei/issues/84#issuecomment-465975392, or mute the thread https://github.com/notifications/unsubscribe-auth/ArnwqMz6aURou5sHSgyBGTpmVfjWdMvpks5vPotZgaJpZM4avvqQ .

mccoys commented 5 years ago

Happi is included in smilei. It is only used for postprocessing, not for running simulations. You simply need to write make happi so that python can find it.

Pint is optional. It is only used in happi, thus does not affect the simulations.

emil-mouz commented 5 years ago

Thanks for the responce.