magland / mountainlab

spike sorting software
25 stars 22 forks source link

Error in bandpass_filter #171

Closed mstux closed 6 years ago

mstux commented 6 years ago

Hi! I get this error when trying to sort my data:

Queuing process from script controller: mountainsort.bandpass_filter

/home/fisio/Scrivania/MountainSort/mountainlab/cpp/mountainprocess/bin/mountainprocess queue-process mountainsort.bandpass_filter --freq_max=6000 --freq_min=300 --freq_wid=1000 --quantization_unit=0 --samplerate=30000 --subsample_factor= --timeseries=/home/fisio/Scrivania/Progetto_prova/bigfiles/prova1/pr2_raw.mda --timeseries_out=/home/fisio/Scrivania/Progetto_prova/mountainlab/tmp_long_term/b4e60660ff116145972fbc3275b099adc98d34f6-mountainsort.bandpass_filter-timeseries_out.tmp --_working_path=/home/fisio/Scrivania/Progetto_prova --_process_output=/home/fisio/Scrivania/Progetto_prova/mountainlab/tmp_short_term/ms.-938054464.140665379592000.476707713.tmp.process_output

mountainsort.bandpass_filter:: STARTING: /home/fisio/Scrivania/MountainSort/mountainlab/packages/mountainsort2/bin/mountainsort2.mp mountainsort.bandpass_filter --timeseries=/home/fisio/Scrivania/Progetto_prova/bigfiles/prova1/pr2_raw.mda --timeseries_out=/home/fisio/Scrivania/Progetto_prova/mountainlab/tmp_long_term/b4e60660ff116145972fbc3275b099adc98d34f6-mountainsort.bandpass_filter-timeseries_out.tmp --freq_max=6000 --freq_min=300 --freq_wid=1000 --quantization_unit=0 --samplerate=30000 --subsample_factor= --_tempdir=/home/fisio/Scrivania/Progetto_prova/mountainlab/tmp_short_term/tempdir_1Q3YRYCHGC . samplerate/freq_min/freq_max/freq_wid: 30000 300 6000 1000 "Unable to allocate Mda of size 1000001x24000x1x1x1x1 (total=24000024000)" ****+++ Using chunk size / overlap size: 20000 / 2000 (num threads=8)

mountainsort.bandpass_filter:: --------------------------------------------------------------- PROCESS COMPLETED (exit code = 255): mountainsort.bandpass_filter ERROR: Exit code is non-zero: mountainsort.bandpass_filter Peak RAM: 0 MB. Peak CPU: 0%. Avg CPU: 0%. Elapsed time: 1,008 seconds.

2017-09-28:12-44-27-522 mp.main critical Error in mountainprocessmain "Exit code is non-zero: mountainsort.bandpass_filter"

mountainsort.bandpass_filter:: Critical: Error in process mountainsort.bandpass_filter N0TW98ABYA69R5ZYD0UQ: Exit code is non-zero: mountainsort.bandpass_filter

Process finished: mountainsort.bandpass_filter

2017-09-28:12-44-28-541 mp.script_controller warning "Process returned with non-zero exit code: mountainsort.bandpass_filter"

Any ideas why? Thank you

magland commented 6 years ago

I think you are running out of RAM. How much RAM to you have on your system?

mstux commented 6 years ago

7,7 GiB

magland commented 6 years ago

What is the size of your raw data? Please try mda /home/fisio/Scrivania/Progetto_prova/bigfiles/prova1/pr2_raw.mda

(how many channels?)

mstux commented 6 years ago

{ "data_type": -7, "data_type_string": "float64", "dims": [ 20000000, 1 ], "header_size": 20, "num_bytes_per_entry": 8, "num_dims": 2 }

Only one channel. I also tried with a smaller file, the error changes but it is still an error:

PROCESS COMPLETED (exit code = 139): mountainsort.bandpass_filter ERROR: Exit code is non-zero: mountainsort.bandpass_filter Peak RAM: 2 MB. Peak CPU: 0%. Avg CPU: 0%. Elapsed time: 7,327 seconds.

2017-09-28:15-54-46-242 mp.main critical Error in mountainprocessmain "Exit code is non-zero: mountainsort.bandpass_filter"

Process finished: mountainsort.bandpass_filter

2017-09-28:15-54-46-442 mp.script_controller warning "Process returned with non-zero exit code: mountainsort.bandpass_filter"

magland commented 6 years ago

You need to transpose the array. It thinks there are 20000000 channels because that is the first dimension. The first dimension is the channel dimension.

mstux commented 6 years ago

Thank you!