Noisy clusters

[shihaisun]

Hi KiloSort team,

A lot of my clusters are noisy following KiloSort spike-sorting. Below is an example of a noisy cluster (as seen in the feature view and waveform view).

I changed the maximum clusters in the config file to 128 and left the rest on default. Can you guys recommend me any setting changes to fix my noisy clustering issue?

Thank you so much for your time, Scott

[nsteinme]

The big vertical lines on the waveform view suggest that you have some very large-amplitude artifacts. You might try detecting these with a simple threshold and then removing that part of the data, then run again. If you don't think that the artifacts impacted your sorting results much, though, then you can simply "split" them out in your phy gui. Looks to me like that would be fine for the example cluster you're showing.

On Sun, Aug 6, 2017 at 6:43 PM, shihaisun notifications@github.com wrote:

Hi KiloSort team,

A lot of my clusters are noisy following KiloSort spike-sorting. Below is an example of a noisy cluster (as seen in the feature view and waveform view).

[image: image] https://user-images.githubusercontent.com/30403651/29005595-ab2b533c-7b21-11e7-980f-5c3bef61e582.png

I changed the maximum clusters in the config file to 128 and left the rest on default. Can you guys recommend me any setting changes to fix my noisy clustering issue?

Thank you so much for your time, Scott

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/cortex-lab/KiloSort/issues/80, or mute the thread https://github.com/notifications/unsubscribe-auth/AHPUP3F3pdwESspPm3c5gvIXMw-u6IdUks5sVfs7gaJpZM4OuyxA .

[brendonw1]

Hey @nsteinme any nice ways you guys have to run such a filter? Maybe a plugin?

[nsteinme]

No, I don't have a method for this - I always run common average referencing before sorting, which I guess must be removing any such artifacts that I have. I rarely see problems like this.

On Fri, Mar 9, 2018 at 2:03 PM, Brendon Watson notifications@github.com wrote:

Hey @nsteinme https://github.com/nsteinme any nice ways you guys have to run such a filter? Maybe a plugin?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cortex-lab/KiloSort/issues/80#issuecomment-371944322, or mute the thread https://github.com/notifications/unsubscribe-auth/AHPUP2cxwoZoW1aRaJS1DgmlBc4spAYXks5tcu4sgaJpZM4OuyxA .

[brendonw1]

Ah i see. That's a good point. So you create a second binary file that's averaged across channels and then run kilosort on that? (also, have you ever tried median filtering?). Thanks

[nsteinme]

Yes, I use this function: https://github.com/cortex-lab/spikes/blob/master/preprocessing/applyCARtoDat.m

No, I haven't tried median filtering...

On Sat, Mar 10, 2018 at 6:06 AM, Brendon Watson notifications@github.com wrote:

Ah i see. That's a good point. So you create a second binary file that's averaged across channels and then run kilosort on that? (also, have you ever tried median filtering?). Thanks

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cortex-lab/KiloSort/issues/80#issuecomment-372028389, or mute the thread https://github.com/notifications/unsubscribe-auth/AHPUPy1kLtJK4YI6X3abOBmXaZHLSHDgks5tc8_SgaJpZM4OuyxA .

[brendonw1]

Thanks! By the way it looks like that does 2D median filtering actually! "% Subtracts median of each channel, then subtracts median of each time % point." (and all calls are "median" not "mean")

[nsteinme]

I think by "median filtering" people sometimes mean like what matlab's "medfilt1" does - where it takes a running window and replaces each point with the median of the values in that window. Here I'm doing something different: First, for each channel separately, I subtract the channel's median (across all time) from every point in the trace (this step just removes channel offsets, which is a thing with neuropixels probes but probably irrelevant for most probes); second, for each time point separately, I take the median across channels and subtract it from every channel's point. The second step is what people call common average referencing (whether it's a mean or median, probably doesn't matter too much...). I should have made that comment in the function more clear!

[g874259]

Hi @nsteinme,

1. Should we do the data preprocessing (subtract channel's median and CAR) ONLY for good channels?
2. The data preprocessing on 1. should be done after high-pass (300Hz) filtering? Tks, JJ

[HankyulKwak]

Hi @nsteinme,

I have a question Firstly, when we start kilosort, it begins with preprocessing. I was wondering if the preprocessing includes applyCARtoDat.m ? Secondly, if it doesn't include applyCARtoDat.m, do I have to run kilosort following applyCARtoDat.m ?

Thanks a lot.

Hankyul Kwak

[nsteinme]

CAR is automatically applied by kilosort! No need for you to take any special step. It will exclude any channels you've labeled as bad before doing the CAR.

On Thu, Aug 26, 2021 at 10:33 PM HankyulKwak @.***> wrote:

Hi @nsteinme https://github.com/nsteinme,

I have a question Firstly, when we start kilosort, it begins with preprocessing. I was wondering if the preprocessing includes applyCARtoDat.m ? Secondly, if it doesn't include applyCARtoDat.m, do I have to run kilosort following applyCARtoDat.m ?

Thanks a lot.

Hankyul Kwak

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cortex-lab/KiloSort/issues/80#issuecomment-906937727, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABZ5IP6NRHJTAQVKTF7SFLLT64PSXANCNFSM4DV3FRAA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.