BUG: filtered signal and prediction not aligned properly in v3.0.1 GUI

teristam commented 5 months ago

Describe the issue:

After upgrading to v3.0.1, the filtered signal and prediction are not aligned to each other in time. The problem goes away when downgraded back to v3.0

Reproduce the bug:

No response

Error message:

No response

Version information:

kilosort: v3.0 Matlab: R2021a

Context for the issue:

No response

Experiment information:

Neuropixel 1.0

marius10p commented 5 months ago

I'll take a look when I have time. 3.0.1 fixes the spike holes bug. Hopefully it did not introduce a time alignment bug.

Could you please double check if this is also a problem for release 2.5.1? The same modification was made there.

teristam commented 5 months ago

Hi,

I think I can also observe a similar thing in v2.5.1:

teristam commented 5 months ago

Just some more updates on the investigation. I am using the first 100s of a recording for testing. The first 2s of data seem to look fine, but the alignment problem starts to appear after the 2.3s point.

marius10p commented 5 months ago

Yeah, there is a difference in how the first batch is treated for the buffer offsets. What I have to determine is if the spike times are correct (and the GUI is wrong) or viceversa. In practical terms it should be a difference of 2-3ms.

zm711 commented 5 months ago

@alejoe91, it looks like this our issue too. I'm just going to copy your message below (from #594) so we can work off this thread instead

@marius10p

zm711 and I noticed that the bug fix branches (at least for KS2.5) is fixing the spike holes, but there seem to be a misalignment with the spike times, so that the extracted waveforms don't look right. See this example: the auto/cross correlograms (bottom right) look correct, but the waveforms are just noise.

Any idea where the misalignment could take place?

alejoe91 commented 5 months ago

I think that the spike times are wrong since the waveforms extracted externally are also wrong. So I suppose that rules out this being a GUI problem

marius10p commented 5 months ago

I have fixed both bugs that were causing the spike holes problem. I think the spike times are correct now, can you please check it in version 3.0.2 (new release)?

teristam commented 4 months ago

I have fixed both bugs that were causing the spike holes problem. I think the spike times are correct now, can you please check it in version 3.0.2 (new release)?

Hello, I haven't tested the new changes in kilosort3 yet because I am trying to get kilosort4 to work. However, I noticed even in the latest version of kilosort4 (4.0.4), the spike time does not seem to align with the index from the BinaryFiltered file (bfile). I assume the index in the bfile corresponds to the actual data index in the original data?

I used the following code to extract spikes directly from the bfile

clu_id = 3
s = []

chan_best = (templates**2).sum(axis=1).argmax(axis=-1)
chan_best = chan_map[chan_best]

cb = chan_best[clu_id]

spk_t = st[clu==clu_id]

for i in range(300):
    t = spk_t[i] 
    s.append(bfile[(t-bfile.nt//2):(t+bfile.nt//2+1),:])
s = np.stack(s)

I noticed the extracted waveforms are mixed:

but when I subtract bfile.nt from the spike time, it works:

And then spike time from the 2nd batch onward don't seem to need this time shift:

Since the spike time is extremely important for analysis, may be more rigorous tests should be implemented?

MouseLand / Kilosort