search

AllenInstitute / ecephys_spike_sorting

Modules for processing extracellular electrophysiology data from Neuropixels probes
Other
109 stars 91 forks source link

Error running mean_waveforms module from curated kilosort output #56

Open edwardyan95 opened 3 years ago

edwardyan95 commented 3 years ago

Hi, I was trying to use the mean_waveforms module and I get the following error towards the end of the analysis.

peak_channels[cluster_idx], IndexError: index 539 is out of bounds for axis 0 with size 539

From what I can tell, it results from the fact that template.npy isn't updated after manual curation in phy, so merging and spliting gives rise to new cluster ids that are not in the original template.npy. The size of peak_channels depends on the size of template (line 87, extract_waveforms.py) but the cluster_ids are from spike_clusters which are changed following curation(line 79, extract_waveforms.py). Currently my workaround is to run the module on the original kilosort output and manually map the original clusters to the manually curated clusters. I'm curious if there's any way that I can run the module on curated kilosort output directly?

Another thing that I noticed is that on line 195 in utils.py, the cluster_quality is inferred from 'group' column of the cluster_group.tsv. However, it seems the kilosort now outputs cluster_group.tsv with 'KSLabel' instead of 'group'.

I really appreciate your work and it has helped a lot with my research, so just want to say thank you!

jsiegle commented 3 years ago

Thanks, this is all helpful to know! Can you share a dataset that's causing these errors? That will make it much easier to troubleshoot.

edwardyan95 commented 3 years ago

Hi, my dataset is really large, so it's difficult to share for now. I will record a short session and then share the data with you. Thanks a lot!

FilippoHei commented 2 years ago

I hav ethe same issue, when running the quality metrics on curated data, due to newly splitted and merged data. Any work-around on that?

edwardyan95 commented 2 years ago

My work around is to add templates for manually added clusters (during curation) to the spike_templates matrix. I can send you the code if you want

moscraciunxxx commented 1 year ago

@edwardyan95 could you share this with me please ? I thought this might be one of the solutions on this matter....

edwardyan95 commented 1 year ago

Hi, sure, you should be able to access it in my public forked repository. I only changed code in main.py file under ../modules/mean_waveforms/

moscraciunxxx commented 1 year ago

@edwardyan95 let ma have a look at it then. Thanks!

moscraciunxxx commented 1 year ago

Hello,

I have modified the PHY’ GUI actual code files and now I got what I needed. 

Let me know if this works for you.

Regards, Vitalie Cervinschi, MSc graduate, Goldsmiths, University of London

Tuesday, 11 July 2023 at 08:07 -07:00 from @. @.>:

My work around is to add templates for manually added clusters (during curation) to the spike_templates matrix. I can send you the code if you want Hi, I have the same issue. It will be greatly helpful if you can share it with me. Thanks! Shivam — Reply to this email directly, view it on GitHub , or unsubscribe . You are receiving this because you commented. Message ID: <AllenInstitute/ecephys_spike_sorting/issues/56/1631003857 @ github . com>

XiaochenFu commented 1 year ago

I got the same issue after splitting the unit clusters in phy. Would there be a easy way to fix this without doing curation again?

edwardyan95 commented 1 year ago

I got the same issue after splitting the unit clusters in phy. Would there be a easy way to fix this without doing curation again?

Please see above comment, you can see my shared repository with an added block of code which should resolve this