espenhgn / ViSAPy

Python package for generating benchmark data for evaluating spike sorting methods
GNU General Public License v2.0
21 stars 8 forks source link

Segmentation Fault 11 #7

Open DominicTanzillo opened 4 years ago

DominicTanzillo commented 4 years ago

I believe we're at the end of step 5 with:

benchmark_data.run() benchmark_data.collect_data()

And when this is running I receive the feedback:

SIZE 1, RANK 0, Cell 0, Min LFP: -0.017, Max LFP: 0.016 Cell 0 saved to file Segmentation fault: 11

Before the terminal crashes. I wanted to see if this is a problem encountered before.

espenhgn commented 4 years ago

Hi @DominicTanzillo, These segmentation faults are usually from NEURON, and I've encountered them more frequently with the latest version (>7.8.1) for some reason. I did not see them in the conda environment I created for your other issue though (which used some older NEURON version).

Can you do another pull of the dev branch, and try again? I added a possible fix (calling cell.__del__() in the Benchmarkdata.cellsim() methods ++).

You can also use MPI to parallelize the simulations across available CPU cores, e.g, mpirun python example_in_vivo_tetrode.py

If you still encounter problems and for the sake of reproducibility, can you provide full details of your configuration?

DominicTanzillo commented 4 years ago

Hello @espenhgn I really appreciate your replies and I've implemented both steps. I'll let you know how the run goes. I'm also using NEURON 7.7.2 which may or may not be the cause. But now that I know the culprit is NEURON, I'll be better able to troubleshoot.

My issue was that the entire console was crashing and not indicating which program would be the problem.

I'll keep you posted.

DominicTanzillo commented 4 years ago

@espenhgn I'm using your .yaml file and and I tried the mpirun. I installed with anaconda and am running an anaconda environment.

This time I received the Fault 11 but with more information:

setup ok! setup ok! inserting 34294 synsapses, TSA : 722957.609 inserting 41929 synsapses, TSA : 923851.374 inserting 13192 synsapses, TSA : 724088.998 inserting 12053 synsapses, TSA : 925148.918

SIZE 2, RANK 0, Cell 0, Min LFP: nan, Max LFP: nan Cell 0 saved to file *** Process received signal *** Signal: Segmentation fault: 11 (11)

When I run from an iPython environment I instead get an infinite error in roughly the same place:

RuntimeWarning: invalid value encountered in true_divide hh = cc / deltaS

But I need to kill the activity to end it.

In both cases it is line 432:

benchmark_data.run() being triggered by a NaN or zero argument being generated by LFP I believe.


For my setup, I've redownloaded your -dev branch. I've uploaded the additional .asc files it prompts me for from an online source.

My computer is a 2016 MacBook Pro. 2.9 GHz Dual-Core Intel Core i5 With 8GB of memory.

Not sure if there is additional information.

espenhgn commented 4 years ago

The Nan's you're seeing is indeed not correct. Did you recompile the .mod files in L5bPCmodelsEH/mod using nrnivmodl from your conda environment? which nrnivmodl should return /Users/<$USER>/anaconda3/envs/<environment-name>/bin/nrnivmodl

DominicTanzillo commented 4 years ago

I've done some troubleshooting here and tried some different combinations. I completed removed and reinstalled NEURON so it is now 7.6 and regenerated the file several ways.

Long story short is that only the mknrndll application drag and drop method words (drag the mod folder onto the mknrndll application or an alias) works.

When running nrnivmodl from the nest216 conda environment in the mod folder, it appears that NEURON does not generate the files properly. When I run the sample file, I immediately run an error letting me know the .hoc files haven't been indexed properly.

If I instead drag and drop the folder onto the application, this exception does not appear.

Doing so with NEURON 7.6 seems to have fixed the problem as I now have figures!

espenhgn commented 4 years ago

Great that you found a work around. I'm not sure what may have been the reason though, perhaps some mismatch between compilers that NEURON may look for and what is available on your system/environment. I never had to use the mknrndll app except for windows related stuff.