Advice for sorting recordings with poor signal-to-noise ratio?

[eroscow]

Hi,

I have some recordings from a 64-channel probe with poor signal-to-noise ratio. In the raw data I can clearly see spikes on most channels, albeit with low amplitude, but Kilosort seems not to be able to detect these and it outputs clusters composed of artefacts and no realistic-looking cells. I have tried adjusting the ops.spkTh spike threshold in the config file, and I have also tried common average referencing the data before inputting it to Kilosort, but to no avail.

Any suggestions for other things I could try?

[PhantomSpike]

Hi,

ops.spkTh will only change the threshold for initializing the spikes for the templates but not the actual detection of events. There are two options for ops.initialize = ''fromData' or 'no'. 'no' uses ready made templates which are in the PCspikes and PCspikes2 .m files. If you select ''fromData' then you use your data to initialize the spikes with the threshold set up in the ops.spkTh. This then uses a k-means clustering to select the templates used for optimisation.

To increase the number of spikes you detect you should change ops.Th and ops.lam. The lower the values, the lower the thresholds. Lowering ops.Th is somewhat equivalent to lowering the numder of std in a simple spike extraction algorithm that only uses thesholding. The lower the values, the more spikes you get. ops.lam controls how much you force the amplitudes/shapes of the detected spikes to be similar to one another for a given cluster. So if the numbers are lower, you are less stringent and ow for more variability and hence more spikes.

If I were you:

1. If your data is noisy, try initialiazing from 'no' rather than 'fromData' as described above.

2. Plot the FFT spectrum for your data. Maybe there is high/low frequency noise that can be filtered out.

3. Are you sure that you are implementing the common average ref correctly? What function are you using?

4. Try decreasing ops.Th and ops.lam to see if you get any more spikes. You will get also more crap but you can just discard it.

5. At the end of the day, there is no substitute for good quality data. Why is the SNR so low? Is it a brain area with very small cells that fire tiny action potentials or it is something to do with the setup, e.g. 50Hz noise, problem with the referencing, ground loops, etc? It's worth investigating and fixing that if possible. I had to toruble-shoot my set-up quite a lot before it worked properly. One thing that would be helpful if you could send me some screenshots of your raw data, e.g. some example traces. What is the rms and std of your signal? With the neuropixels the signal should not be more than 8-12 uV rms/std in saline and slightly higher than that in the brain.

6. One thing you could do for sanity checking. If you can literally see the spikes with the naked eye, then try doing the CAR, then filter the data between 300 and 3,000Hz, compute the STD of your signal and then apply a simple thresholding value like Th = 3STD and call any event that crosses that Th a spike. Can you see your spikes now? Can you get any sort of meaningful PSTHs, tuning curves etc? If yes, than it's worth playing with Kilosort. If not, then you have a serious problem and maybe what you are seeing are not really spikes. Have some functions for that, let me know if you need any help.

Alex

[Alchemist-Y]

@PhantomSpike Hi, thanks for your minute illustration, which is really beneficial for the use of Kilosort and the spike sorting pipline! I don't know if you follow the latest version of Kilosort4 or not, I proposeed an issue where it seems I meet the same problem. So I want to exchange some details with you in order to get your great advice!

1. I havn't found the "There are two options for ops.initialize = ''fromData' or 'no'" options, maybe it is deleted in the latest kilosort version?
2. I wonder that if you try to narrow down the ops.Th and ops.lam which will help you catch more spikes and clusters, but is this number increasing will be benefit for the low SNR? it seems no relationship?
3. Could these preprocess such as doing the CAR or filter the signal with its STD you mentioned above be done in kilosort4? It seems you did the preprocess before and without Kilosort. If the signal still shows spikes after these preprocess, should I load the raw data to Kilosort directly or load the preprocessed data into it? Thanks a lot in advance!

[jonahpearl]

1. I havn't found the "There are two options for ops.initialize = ''fromData' or 'no'" options, maybe it is deleted in the latest kilosort version?

Hi Alchemist-Y — I had a similar problem to you on some noisy data sets. It's the templates_from_data setting in ks4: https://github.com/MouseLand/Kilosort/blob/298a0864f88842ef7964625536a258b09f491ba9/kilosort/parameters.py#L263. I am surprised this isn't mentioned in the KS4 docs as I see much better results on mildly noisy recordings when I use the pre-computed templates. There is still noise but is clear in Phy which is which.

[Alchemist-Y]

It's the templates_from_data setting in ks4:

Thanks for your minute help! Here is my question: how do you define "midly" nosie? You means you calculate the SNR? Actural the noise do will influrence the firing rate, so I'm now trying to avoid the noise as much as possible with the preprocessing session.

[jonahpearl]

I guess I mean something similar to what PhantomSpike described above — basically, you can see some spikes by eye and convince yourself that the situation improves after pre-processing.

[Alchemist-Y]

@jonahpearl Yes I agree with it

[Alchemist-Y]

@jonahpearl But I stil get some concern: KS4 do will make some preprocessing such bandpass filter, whiten and CAR, is it necessary to do it again or more preprocessing operation parts to kill the noise in the spikeinterface?