Units table raster plot

[bendichter]

Looking through datasets such as https://dandiarchive.org/dandiset/000582 it is clear that we would benefit from a units raster plot. Like the time series traces viewer, modern probes can record from many units at a time, so we need a way to only show a subset and buttons to pan through the units.

[magland]

@bendichter

A prelim raster plot is available -- expand the units panel, and you'll see links for autocorrelograms and raster plot

These views require data prepared and saved in the external bucket. If not already computed, you can click to request that it be created. Wait for a bit and then refresh the units panel.

There is a backend service that is listening for and fulfilling requests - but it is not required once the computed data are in the external bucket.

We should chat some more about this...

[bendichter]

@magland I feel like you should be able to just read all of the spiking data directly. Happy to chat more about this.

In any case, could you add a time slider controller like you have for TimeSeries plots? It follows the time slider if present in another plot, but when viewed alone you can't jump to a specific time.

[magland]

@magland I feel like you should be able to just read all of the spiking data directly. Happy to chat more about this.

That works if the dataset is small. But I would really like to handle datasets of all sizes using the same system. The way that the data are saved in the nwb, there is no way to lazily load time chunks -- you need to first download all the units data to the browser.

Here's an example of 68 units over ~16 hours. The .ns-spt format allows efficient random access.

https://flatironinstitute.github.io/neurosift/?p=/nwb&url=https://dandiarchive.s3.amazonaws.com/blobs/c86/cdf/c86cdfba-e1af-45a7-8dfd-d243adc20ced

In any case, could you add a time slider controller like you have for TimeSeries plots? It follows the time slider if present in another plot, but when viewed alone you can't jump to a specific time.

I'll open a separate issue for this.

[magland]

Here's another example of a neuropixels probe over 2.5 hours. For the raster and AC, this is too much data to load directly from the nwb file.

https://flatironinstitute.github.io/neurosift/?p=/nwb&url=https://dandiarchive.s3.amazonaws.com/blobs/544/745/5447455e-49ec-4ce0-9a64-4cad767bef7b

[bendichter]

OK, but it should be pretty easy to sub-select to <=30 units, which would substantially reduce the amount of data you need to load at once. So this would only ever be an issue for extremely long recordings or if the user is viewing many neurons at once. If we restrict the number of units they are able to view at once, we should be able to get this to work, right?

[magland]

I see what you're saying. I think we're going to want to have both.

Here's an example with 684 units, and you can look for correlations between stimulus and raster plot

That wouldn't be practical reading directly from the nwb file. For reference, it's this dataset: https://dandiarchive.org/dandiset/000473/0.230417.1502/files?location=sub-128514

On the other hand, for smaller datasets, it's simpler if we can just download from nwb all at once. I'll implement that for the smaller examples.

The annoying case, from an engineering perspective, is wheree we want to restrict to a subset of units. How would you imagine that working in the GUI?

[bendichter]

I imagine it the same way we do it with the time series traces

[bendichter]

You're right, it is nice to be able to look at hundreds of units at once

[magland]

@bendichter here's what I have for now.

There are a few view plugins for the Units neurodata type, including 'raster plot' and 'precomputed raster plot'

The regular raster plot reads directly from the nwb file, but limits to viewing only a subset of units if there are too many spikes. For now there's no way to specify which units to display.

The precomputed raster plot can render all the units no matter how large the dataset. If the precomputed file does not exist, it can be requested in the GUI.

Similar for precomputed autocorrelograms - I don't plan on implementing autocorrelograms on the fly (we can discuss).

Here are a couple examples. This first one is small enough that the regular raster plot works fully:

https://flatironinstitute.github.io/neurosift/?p=/nwb&url=https://dandiarchive.s3.amazonaws.com/blobs/a63/6de/a636de8b-7c90-4a41-94be-9da3de53cf82

This second one is much larger, so you'll want to view the precomputed one.

https://flatironinstitute.github.io/neurosift/?p=/nwb&url=https://dandiarchive.s3.amazonaws.com/blobs/544/745/5447455e-49ec-4ce0-9a64-4cad767bef7b