INM-6 / hybridLFPy

Biophysics-based prediction of LFPs from point-neuron networks
GNU General Public License v3.0
10 stars 10 forks source link

segmentation fault when running multi-compartment simulation #41

Closed HilbertHuangHitomi closed 3 years ago

HilbertHuangHitomi commented 3 years ago

Hi~ I would like to use cellsim16pops_*.py to simulate. It works well during the LIF network simulation and processing the raw nest output, but fails to simulate the multi-compartment models.

Here is my scheme.

  1. create an environment with conda create --name neuro python=3.8
  2. install necessary packages LFPy==2.1.2, neuron==7.8.2, hybridLFPy==0.1.4
  3. compile mod mechanisms with nrnivmodl
  4. run the simulation with mpiexec python cellsim16pops_default.py or mpirun -np 256 python cellsim16pops_default.py

Both the LIF network simulation and the post-processed part of the multi-compartment simulation works well and gives expected results to the folder.

However, after reporting population initialized in X seconds, it raises Segmentation fault as

[SERVER:02380] Process received signal [SERVER:02380] Signal: Segmentation fault (11) [SERVER:02380] Signal code: Address not mapped (1) [SERVER:02380] Failing at address: 0x8 [SERVER:02380] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7fe2a75b9390] [SERVER:02380] [ 1] /user/miniconda3/envs/neuro/lib/python3.8/site-packages/neuron/.data/lib/libnrniv.so(_ZN8NetCvode11init_eventsEv+0x1eb)[0x7fe2232810eb] [SERVER:02380] [ 2] /user/miniconda3/envs/neuro/lib/python3.8/site-packages/neuron/.data/lib/libnrniv.so(nrn_finitialize+0x3f5)[0x7fe2232d98e5] [SERVER:02380] [ 3] /user/miniconda3/envs/neuro/lib/python3.8/site-packages/neuron/.data/lib/libnrniv.so(finitialize+0x3a)[0x7fe2232d9aea] [SERVER:02380] [ 4] /user/miniconda3/envs/neuro/lib/python3.8/site-packages/neuron/.data/lib/libnrniv.so(hoc_call+0x18e)[0x7fe2232d505e] [SERVER:02380] [ 5] /user/miniconda3/envs/neuro/lib/python3.8/site-packages/neuron/.data/lib/libnrnpython3.so(+0x13ff8)[0x7fe222defff8] [SERVER:02380] [ 6] /user/miniconda3/envs/neuro/lib/python3.8/site-packages/neuron/.data/lib/libnrniv.so(ZN10OcJumpImpl7fpycallEPFPvS0_S0_ES0_S0+0x3e)[0x7fe2232a82ee] [SERVER:02380] [ 7] /user/miniconda3/envs/neuro/lib/python3.8/site-packages/neuron/.data/lib/libnrnpython3.so(+0x1418e)[0x7fe222df018e] [SERVER:02380] [ 8] python(_PyObject_MakeTpCall+0x31e)[0x55a826c95f2e] [SERVER:02380] [ 9] python(_PyEval_EvalFrameDefault+0x534b)[0x55a826d2ff6b] [SERVER:02380] [10] python(_PyEval_EvalCodeWithName+0x2c3)[0x55a826d0b503] [SERVER:02380] [11] python(_PyFunction_Vectorcall+0x378)[0x55a826d0c8d8] [SERVER:02380] [12] python(+0x1b32a7)[0x55a826d0e2a7] [SERVER:02380] [13] python(+0x124fba)[0x55a826c7ffba] [SERVER:02380] [14] python(PyObject_Call+0x255)[0x55a826c802b5] [SERVER:02380] [15] python(_PyEval_EvalFrameDefault+0x21c1)[0x55a826d2cde1] [SERVER:02380] [16] python(_PyEval_EvalCodeWithName+0x2c3)[0x55a826d0b503] [SERVER:02380] [17] python(_PyFunction_Vectorcall+0x378)[0x55a826d0c8d8] [SERVER:02380] [18] python(_PyEval_EvalFrameDefault+0xa4b)[0x55a826d2b66b] [SERVER:02380] [19] python(_PyFunction_Vectorcall+0x1a6)[0x55a826d0c706] [SERVER:02380] [20] python(_PyEval_EvalFrameDefault+0xa4b)[0x55a826d2b66b] [SERVER:02380] [21] python(_PyEval_EvalCodeWithName+0x2c3)[0x55a826d0b503] [SERVER:02380] [22] python(PyEval_EvalCodeEx+0x39)[0x55a826d0c559] [SERVER:02380] [23] python(PyEval_EvalCode+0x1b)[0x55a826daf9ab] [SERVER:02380] [24] python(+0x254a43)[0x55a826dafa43] [SERVER:02380] [25] python(+0x26e6b3)[0x55a826dc96b3] [SERVER:02380] [26] python(+0x2735b2)[0x55a826dce5b2] [SERVER:02380] [27] python(PyRun_SimpleFileExFlags+0x1b2)[0x55a826dce792] [SERVER:02380] [28] python(Py_RunMain+0x36d)[0x55a826dced0d] [SERVER:02380] [29] python(Py_BytesMain+0x39)[0x55a826dceec9] [SERVER:02380] End of error message Primary job terminated normally, but 1 process returned a non-zero exit code. Per user-direction, the job has been aborted. mpiexec noticed that process rank 14 with PID 0 on node SERVER exited on signal 11 (Segmentation fault).

I have tried to fix this with install neuron 8.0a following https://github.com/neuronsimulator/nrn/pull/1010, downloading the wheel and running pip install NEURON-8.0a0-cp38-cp38-manylinux1_x86_64.whl in a new environment, but the error still rises.

Could you help me to solve it, please?

espenhgn commented 3 years ago

Hi. I think I need more info in order to try and reproduce this crash. Can you post the output of conda info and conda list? Which what is the git SHA for your NEST installation? When starting NEST (nest on the command line may print out Version: master@8f5a5fc1e or similar).

espenhgn commented 3 years ago

Hi again. If you're using conda my recommendation is to use as many dependencies as possible from conda-forge in order to avoid issues with slightly incompatible versions of gcc etc. between NEST and NEURON. Whatever was used to compile NEURON for PyPi.org may be different to what you used with NEST compiled locally. I could run the simplest case example_brunel.py by the following procedure:

conda create -n hybridlfpy python=3.8 lfpy libtool gsl doxygen cmake -c conda-forge
conda activate hybridlfpy

# compile nest
git clone https://github.com/nest/nest-simulator.git
cd nest-simulator
git checkout 8f5a5fc1e   # this is the version I've been using. Things may have changed upstream lateron. 
mkdir nestbld
cd nestbld
cmake -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX -Dwith-mpi=ON ..
make -j4
make install

cd ../..

cd hybridLFPy
<git checkout nest3>  # make sure to use the origin nest3 branch 
python setup.py install
cd examples
pip install git+https://github.com/NeuralEnsemble/parameters
nrnivmodl
mpirun -np 2 python example_brunel.py

should then hopefully work.

Also, please keep in mind that the microcircuit examples are perhaps best executed on HPC resources due to long simulation times on typical laptops/desktops. On HPC/compute clusters using conda may not be the best idea as you're better off most likely to compile both NEURON and NEST using the system compilers and libraries which may be better optimized.

HilbertHuangHitomi commented 3 years ago

All works well following your suggestions. Thank you very much! (≧∇≦)ノ

espenhgn commented 3 years ago

Great!