search

cortex-lab / phy

phy: interactive visualization and manual spike sorting of large-scale ephys data
BSD 3-Clause "New" or "Revised" License
313 stars 157 forks source link

Co-Curing multiple sessions worth of data #1096

Open achristensen56 opened 3 years ago

achristensen56 commented 3 years ago

Hi!

I was wondering if anyone has any cute tricks as far as trying to use Phy to cure data that has been clustered across multiple sessions. For example, is there an easy way to add some metadata such that waveforms, clusters, etc. from session1 (defined by spike timestamp, e.g.) are displayed in a different color than the data from session2? I'm looking at the customization section of the docs and considering writing a widget for this, but before I dive in I was wondering if there are easy existing ways to use phy for this purpose.

what would be great is if we could have two windows, similar to clusters and similar clusters, which corresponded to either two separately sorted datasets, or that corresponded to two separate time spans of a single sorted dataset.

thanks! Amy

czuba commented 3 years ago

Commenting/subscribing to say that this is something I would also love to see developed & would be very useful; either as a plugin or full-fledged multi-session feature in Phy. ...particularly, as you mentioned, for files that require running split subsets to be run through kilosort, then post-hoc merged manually —e.g. files with an abrupt spatial (probe shift) or temporal (time gap) discontinuity.

@achristensen56 curious if you've tried/succeeded in manually concatenating the output files from separate kilosort sessions, then loading the merged set into Phy? It wouldn't be as pretty as the true Phy extension you've proposed, but the session assignment should be obvious in the amplitude view.

[...even a simple "nah, didn't try it" or "yeah, but it was a nightmare rabbit-hole" would be much appreciated.]

In either case, in addition to combining clusters with [subjectively] coherent waveforms from different sorting sessions, this kind of 'merged Phy session' would be great for pooling low-threshold clusters from a local region (channel, adjacent pair, row of sites, whatever) into a common multi-unit cluster across sorting sessions.

...the sort of cluster merging that traditionally occurred during hand-sorting spikes into a distinctly-not-noise—but-also-not-separable-clusters MUA group that would otherwise fall through the cracks when applying a post-hoc "mergeSessionSingleUnits" function that only considered mean waveforms for matching clusters across sorting sessions.

achristensen56 commented 3 years ago

I actually never got around to implementing that! We decided to cure the individual days from session1 and session2 then match neurons in the co-sorted session, accepting a neuron if it was accepted in both constituent days.

But also curious if you try the concatenation how it goes!

edit @czuba I'm interested in implementing a pipeline for most of the issues you described, and I'm going to be working on it in the coming few weeks. let me know if you want to work together on this and maybe make it publically available -- if all the work is going to go into it, mineaswell help other people.