search

flatironinstitute / ironclust

Spike sorting software being developed at Flatiron Institute, based on JRCLUST (Janelia Rocket Cluster)
Apache License 2.0
28 stars 7 forks source link

Out of memory issues when trying to export to "Klusters/neurosuite" format for long recordings #21

Closed bapungiri closed 5 years ago

bapungiri commented 5 years ago

I am getting the following error when trying to export the files into neurosuite format. This error was not encountered for sample data but happens for long recordings (>14 hrs). Other details: ironclust version 4.8.10

irc export-klusters Shank3-4.prm loading Shank3-4jrc.mat... took 1.5s Computing templates per unit In /home/bapung/Documents/MATLAB/ironclust/matlab/irc.m>disperr (line 18291) In /home/bapung/Documents/MATLAB/ironclust/matlab/irc.m>fread_ (line 5346) In /home/bapung/Documents/MATLAB/ironclust/matlab/irc.m>loadbin (line 2247) In /home/bapung/Documents/MATLAB/ironclust/matlab/irc.m>load_binT (line 30524) In /home/bapung/Documents/MATLAB/ironclust/matlab/irc.m>load_wavmed (line 30555) In /home/bapung/Documents/MATLAB/ironclust/matlab/irc.m>test (line 19032) In /home/bapung/Documents/MATLAB/ironclust/matlab/irc.m>call (line 19066) In /home/bapung/Documents/MATLAB/ironclust/matlab/irc.m>irc (line 131) In /home/bapung/Documents/MATLAB/ironclust/matlab/irc2klusters.m>irc2klusters (line 40) In /home/bapung/Documents/MATLAB/ironclust/matlab/irc.m>irc (line 178) Error log updated: /home/bapung/Documents/MATLAB/ironclust/matlab/errorlog.mat Error using fread Out of memory. Type "help memory" for your options. GPU device reset In /home/bapung/Documents/MATLAB/ironclust/matlab/irc.m>disperr (line 18291) In /home/bapung/Documents/MATLAB/ironclust/matlab/irc.m>loadbin (line 2253) In /home/bapung/Documents/MATLAB/ironclust/matlab/irc.m>load_binT (line 30524) In /home/bapung/Documents/MATLAB/ironclust/matlab/irc.m>load_wavmed (line 30555) In /home/bapung/Documents/MATLAB/ironclust/matlab/irc.m>test (line 19032) In /home/bapung/Documents/MATLAB/ironclust/matlab/irc.m>call (line 19066) In /home/bapung/Documents/MATLAB/ironclust/matlab/irc.m>irc (line 131) In /home/bapung/Documents/MATLAB/ironclust/matlab/irc2klusters.m>irc2klusters (line 40) In /home/bapung/Documents/MATLAB/ironclust/matlab/irc.m>irc (line 178) Error log updated: /home/bapung/Documents/MATLAB/ironclust/matlab/errorlog.mat Output argument "mnWav1" (and maybe others) not assigned during call to "irc>fread". GPU device reset In /home/bapung/Documents/MATLAB/ironclust/matlab/irc.m>disperr (line 18291) In /home/bapung/Documents/MATLAB/ironclust/matlab/irc.m>test (line 19048) In /home/bapung/Documents/MATLAB/ironclust/matlab/irc.m>call_ (line 19066) In /home/bapung/Documents/MATLAB/ironclust/matlab/irc.m>irc (line 131) In /home/bapung/Documents/MATLAB/ironclust/matlab/irc2klusters.m>irc2klusters (line 40) In /home/bapung/Documents/MATLAB/ironclust/matlab/irc.m>irc (line 178) Error log updated: /home/bapung/Documents/MATLAB/ironclust/matlab/errorlog.mat Index in position 2 is invalid. Array indices must be positive integers or logical values. GPU device reset Output argument "varargout{2}" (and maybe others) not assigned during call to "irc>call".

Error in irc (line 131) case 2, [varargout{1}, varargout{2}] = call_(vcArg1, vcArg2);

Error in irc2klusters (line 40) [mrWav_clu1, mnWav_T] = irc('call','load_wavmed', {P1, viTime_spk1, mnWav_T}); % not filtering

Error in irc (line 178) case {'export-klusters', 'klusters', 'neurosuite'}, irc2klusters(vcFile_prm, vcArg2);

jamesjun commented 5 years ago

I will work on a paged loading scheme to address this issue.

jamesjun commented 5 years ago

Resolved. I also implemented an exporter for phy. Usage: ‘irc export-klusters myparam.prm’. For phy, use ‘export-phy’ command.

jamesjun commented 5 years ago

Resolved. I also implemented an exporter for phy. Usage: ‘irc export-klusters myparam.prm’. For phy, use ‘export-phy’ command.