search

cortex-lab / KiloSort

GPU code for spike sorting
GNU General Public License v2.0
179 stars 100 forks source link

Error using latest npy-matlab version for phy export #185

Closed hugoladret closed 4 years ago

hugoladret commented 5 years ago

Hi,

First of all, thank you very much for developping KiloSort, it's a fantastic tool that saves me a great deal of time compared to CPU-based spike sorting softwares.

I've however ran into a slight problem when saving the rez files using the latest version of npy-matlab. KS correctly saves rez as .npy files, but there seems to be a problem in the header, as numpy cannot open the file(either using phy or command-line python). image image Opening the file with text editors or with Python file.readlines() doesn't work either, neither does changing numpy version.

Using npy-matlab version https://github.com/kwikteam/npy-matlab/commit/524bd143c34cbd9d1bbc895ed05e082ce4249b62 does fix the problem.. perhaps this should be included in the docs ? If this is a previously unreported issue i'd be happy to provide more detailled logs.

Cheers, Hugo

nsteinme commented 5 years ago

Hmm. I don't think there's any difference in the header construction between those two versions of npy-matlab...? Could you upload the two versions of channel_map.npy, the one that works and the one that doesn't? Are you sure the one that doesn't work uses the latest npy-matlab? I have been using a fresh install on a new computer with kilosort recently and have not encountered this problem. P.S. it's best if you go ahead and try Kilosort2 - https://github.com/MouseLand/Kilosort2 - and see whether you still have the problem there, too.

On Wed, Mar 20, 2019 at 7:41 AM Hugo Ladret notifications@github.com wrote:

Hi,

First of all, thank you very much for developping KiloSort, it's a fantastic tool that saves me a great deal of time compared to CPU-based spike sorting softwares.

I've however ran into a slight problem when saving the rez files using the latest version of npy-matlab. KS correctly saves rez as .npy files, but there seems to be a problem in the header, as numpy cannot open the file(either using phy or command-line python). [image: image] https://user-images.githubusercontent.com/16063162/54692732-dad2b700-4afb-11e9-8752-e519162b61e2.png [image: image] https://user-images.githubusercontent.com/16063162/54692965-37ce6d00-4afc-11e9-918a-08a5d749c300.png Opening the file with text editors or with Python file.readlines() doesn't work either, neither does changing numpy version.

Using npy-matlab version kwikteam/npy-matlab@524bd14 https://github.com/kwikteam/npy-matlab/commit/524bd143c34cbd9d1bbc895ed05e082ce4249b62 does fix the problem.. perhaps this should be included in the docs ? If this is a previously unreported issue i'd be happy to provide more detailled logs.

Cheers, Hugo

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/cortex-lab/KiloSort/issues/185, or mute the thread https://github.com/notifications/unsubscribe-auth/AHPUPyec4SUr4HBqvSZUNmHAhzQsSZQZks5vYkiGgaJpZM4b_eCy .

hugoladret commented 5 years ago

I don't see a major difference between commits either, which is why this 'fix' took me some time to find..

Here are two amplitudes.npy files, as the bug really happens in all .npy file and amplitudes is the first one to be loaded by phy template-gui : https://we.tl/t-swKSHGM5AY
amplitudes_new_npy is produced with the most recent npy-matlab, while amplitudes_older_npy is produced with the version 524bd14 and is the one that can be loaded into phy. Both files come from running the eMouse set.

As for moving to KS2 I definitely will, but bumped into nondescriptive CUDA memory error and have yet to try using another GPU to see where that comes from :-)

hugoladret commented 4 years ago

Cant replicate in KS2 and in newer versions of KS1, closing.