search

flatironinstitute / mountainlab-js

MountainLab is data processing, sharing and visualization software for scientists. It is built around MountainSort, spike sorting software, but is designed to be more generally applicable.
Other
43 stars 30 forks source link

ms4alg error with intermediate files #25

Closed izouridis closed 6 years ago

izouridis commented 6 years ago

Hello mountainlab-js developers,

I'm trying to use ms4alg with my data and I get this error: KeyError: "Unable to open object (file read failed: time = Tue Jun 19 16:04:17 2018\n, filename = '/media/C/Projects/SPIKE_SORT/temp/tempdir_a072851555_xIrCyD/timeseries.hdf5', file descriptor = 4, errno = 22, error message = 'Invalid argument', buf = 0x2f0a168, total read size = 544, bytes this sub-read = 544, bytes actually read = 18446744073709551615, offset = 5405935000)"

Here is the full output: https://pastebin.com/LP8qf5wC

Best, Ioannis

alexmorley commented 6 years ago

Hey, could you just confirm that this happens every time you sort on this dataset? And/or on other datasets?

izouridis commented 6 years ago

This happens with most of my datasets (11/12). There is one dataset that seems to work fine and it happens to be the smallest one.

alexmorley commented 6 years ago

Could you let us know how much RAM you have. The size of the dataset that doesn't work. And how much space you have in /media/C/?

izouridis commented 6 years ago

RAM allocated to the VM: 106.466 GB. Free space in /media/C: >640 GB. Size of dataset (raw.mda) that doesn't work: 39.9 GB Size of intermediate file: 19.7 GB

magland commented 6 years ago

@izouridis. Sorry I just noticed this issue after several weeks. Could you confirm that the problem still exists, and if so, can we arrange for you to share the dataset via kbucket so we can debug together?

izouridis commented 6 years ago

@magland this problem has been fixed. I close this issue. Thank you.