using mounainsort for spike snippets

[LaurenzMuessig]

Hi,

I am using mountainsort3 (the default pipeline) to sort data from a thresholded tetrode recording, so I only have 1ms snippets for all potential events. For the sorting I create a 'mock' continuous trace where I put all spike snippets at the exact time point in the trial they were recorded (padded by zeros where no spike occurred). Now, when I run mountainsort there are a few issues that I don't fully understand: i) when I run it without the --curate-true flag it does find more spikes/events than I have fed in (I set clip_size_msec=1 and detect_interval_msec=1 as someone from L.Franks lab suggested on GitHub) ii) the peak indices that are returned in firings_out are not exactly the actual sample positions of the peak in the mock trace - they tend to vary between ~1-10 samples from the data trace (the firings_out is based on 1-based indexing correct?)?

I understand mountainsort is probably (like kilosort) more designed to work on continuously recorded data, but if you have any idea why this happens when using it on snippets I would appreciate your opinion E.g. is it a problem for the data sorting that the data trace contains a lot of 0's.

Ps: I am also struggling to find a full list of all parameters I can pass in the parameters file - does such a list exist somewhere?

Thanks for your help.

Best

Laurenz

[gaelias]

Hi Laurenz,

I'm also using snippets (8ms) and struggled to get MountainSort to play well with my recreated continuous data. I took the zero pad approach (it does not like NaNs at all) but found that the main "units" that was being pulled out were the edges of my snippets where the voltage suddenly jumped up/down to zero. I switched over to instead, connecting the end of one trace with the beginning of the next with a straight line and once I made that change it cleaned things up immensely.

From my experience I can also suggest you always use --curate=true. From my data I've found that when I omit this MountainSort will double count waveforms (it still does it even after curation, but to a much smaller degree). Sometimes it will double count the waveform and associate it with the same template with a very short ISI, or it will double count the waveform and associate it with different templates. I haven't played around with the clip_size_msec and detect_interval_msec variables, so specifying them in my params file may clear up this issue for me but it's something to be wary of.

Regarding parameters, I haven't found anything like a formal list of them but I used the mp-spec command to get the notes on all the processors and then formatted (most of) that output as a word document. It gives you a list of the processors and their input/output/parameters and most of them are well described.

Best, -gabe

[magland]

Hi Laurenz, and thanks Gabe. I have not tried with snippets, so it's hard for me to give advice, or help track down the issues. But perhaps we can set up a time to chat (if you still have questions)? Maybe that will be easier for this situation. Feel free to contact me directly or on slack.

[LaurenzMuessig]

Hi Gabe & Jeremy,

Thanks for your exhaustive answer and sorry for the late reply. My snippets are actually a lot shorter, only 1ms, so they really only capture the spike waveform. I have tried adding some 'mock' noise in between my snippets - I want to keep the temporal relationship between all snippets so that in the end I can get a meaningful autocorrelation for each cluster. I will try your approach with using a line

From my experience I can also suggest you always use --curate=true. I noticed that this improves stuff immensely too and this is what I would typically use. However, I wanted to check that the sorting doesn't return more events than I had 'put' in. I have also tried using kilosort and the recent Janelia Rocket clust and I find that basically all algorithms seem to have an issue with thresholded tetrode recordings (well, at least mine). If I cluster 1 tetrode then even though a spike gets recorded on all 4 channels at the same time point the signal might get assigned to several clusters which is why in the end there are more events than I actually recorded. I haven't tested this exhaustively but mountainsort seems to perform best, i.e. have the least amount of this 'double' counting.

I am very busy at the moment with preparing a manuscript but once I can get back to fiddling with mountainsort it would be great to have a chat with you Jeremy (and Gabe if you are interested). How can I find you on slack? I need to do a bit more testing so I will have some meaningful question. I'll be in touch and thanks for all your help guys.

Best

Laurenz

[gaelias]

Hi Laurenz,

Even with curation I still find that waveforms are being duplicated and assigned to multiple templates. In some instances the waveform is a compound of two templates that happen near simultaneously, other times it's an ambiguous waveform. I'm using MountainSort as a first pass and use OfflineSorter to clean up the clusters and cut the ones the algorithm misses. When I convert the mda files into an OFS friendly format I add in any of my original snippets that MountainSort ignored and identify any double counted waveforms and dump all those in as unassigned waveforms (I only take one copy of the duplicate waveforms otherwise OFS can't designate the four channels as tetrodes recordings).

When I first was playing around with things and examining the uncurated cuts I found that there were many duplicate assignments that fell off center to the obvious waveform peak. Aside from using curate, the other things that cleaned thing up (not entirely, curate is still the best) was making sure my params file had the correct filter settings and setting adjacency ratio to -1.

I would be interested in joining the conversation if you start one on slack.

Best, -gabe

[magland]

regarding slack, send me a direct email with your request, and I'll invite you to the slack channel.

[x8808099]

Hi,

I'm trying to make ms3 work with snippets, do you come up with a better solution now?

Thanks -peiyao

[magland]

We are working on it. Are you on the slack site (if not, request access by sending me an email)? Post your question there, and I will put you in contact with the other folks who are trying to use snippets data with MountainSort.

[gaelias]

Hi Peiyao,

My snippets are recorded from a Plexon system so all the files are in .plx format. I wrote some matlab code that constructs a continuous trace from the snippets and saves them in .mda format to run through the standard MountainSort pipeline. I also have code written that reconstructs the firings output into a .NEX (NeuroExplorer) file that I can open in Plexon's OfflineSorter program and convert them back into .plx so I can do manual curation with OfflineSorter as my final step. I do the manual curation on the back end because MountainSort will sometimes think the ends of the snippets are spikes so I have to take them out manually. The code is up in our lab's GitHub page if you want: https://github.com/FortinLab/MSssInteg

Best, -gabe

[stoneqq]

regarding slack, send me a direct email with your request, and I'll invite you to the slack channel. We are working on it. Are you on the slack site (if not, request access by sending me an email)? Post your question there, and I will put you in contact with the other folks who are trying to use snippets data with MountainSort.

Hi Dr. Magland, Thank you for developing such great software! I am new to mountainsort and I also ran into similar issues when trying to figure it out. I was able to get the software running following the Readme instructions and issue discussions, but I am not sure how to interpret the parameters setting and result formats since there is not enough information to refer to. I've sent you an email request to your Gmail and flatiron mail under the name of Delin, hope you are still using those email address. If you could invite me to the Slack channel so I can follow up with the discussions. I can help with the documentation once I figure it out. Thank you!

Thanks, Delin

[magland]

Hi Delin. I sent you an invite. Please also see the following under development: https://github.com/flatironinstitute/spikeforest2 This provides wrappers to a number of spike sorters, including MountainSort. It is under development. Jeremy

On Thu, Feb 21, 2019 at 12:15 PM Ben notifications@github.com wrote:

regarding slack, send me a direct email with your request, and I'll invite you to the slack channel. We are working on it. Are you on the slack site (if not, request access by sending me an email)? Post your question there, and I will put you in contact with the other folks who are trying to use snippets data with MountainSort.

Hi Dr. Magland, Thank you for developing such great software! I am new to mountainsort and I also ran into similar issues when trying to figure it out. I was able to get the software running following the Readme instructions and issue discussions, but I am not sure how to interpret the parameters setting and result formats since there is not enough information to refer to. I've sent you an email request to your Gmail and flatiron mail under the name of Delin, hope you are still using those email address. If you could invite me to the Slack channel so I can follow up with the discussions. I can help with the documentation once I figure it out. Thank you!

Thanks, Delin

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/flatironinstitute/mountainsort/issues/16#issuecomment-466085240, or mute the thread https://github.com/notifications/unsubscribe-auth/ADgkQLuAC14F1khlkmwueXL0VjDJoWhiks5vPtQ4gaJpZM4Q5lNi .

[stoneqq]

Hi Delin. I sent you an invite. Please also see the following under development: https://github.com/flatironinstitute/spikeforest2 This provides wrappers to a number of spike sorters, including MountainSort. It is under development. Jeremy … On Thu, Feb 21, 2019 at 12:15 PM Ben @.***> wrote: regarding slack, send me a direct email with your request, and I'll invite you to the slack channel. We are working on it. Are you on the slack site (if not, request access by sending me an email)? Post your question there, and I will put you in contact with the other folks who are trying to use snippets data with MountainSort. Hi Dr. Magland, Thank you for developing such great software! I am new to mountainsort and I also ran into similar issues when trying to figure it out. I was able to get the software running following the Readme instructions and issue discussions, but I am not sure how to interpret the parameters setting and result formats since there is not enough information to refer to. I've sent you an email request to your Gmail and flatiron mail under the name of Delin, hope you are still using those email address. If you could invite me to the Slack channel so I can follow up with the discussions. I can help with the documentation once I figure it out. Thank you! Thanks, Delin — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#16 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/ADgkQLuAC14F1khlkmwueXL0VjDJoWhiks5vPtQ4gaJpZM4Q5lNi .

Hi Dr. Magland, Thank you so much for the invite! I will definitely check out the spikeforest. Delin

[cspurandare]

hello, I am trying to make the snippet version of Mountainsort work. (Neuralynx recording, 1.5ms pre-detected spikes). I have followed the suggestions here of padding with interpolation, and played around with clip-size, detection threshold and detect-interval parameters, but I am not sure what other parameters could improve clustering. Has there been any conclusion/agreement about the best way to go ahead for snippets. I was hoping you could please add me to the Slack channel as well? Thanks and warm regards, Chinmay