ndx-microscopy needs follow-up

[CodyCBakerPhD]

@magland After #204 was merged, I still do not see any Neurosift plugins associated with the data types

E.g.: https://neurosift.app/?p=/nwb&url=https://api.dandiarchive.org/api/assets/e6263508-26a0-4db0-ac0a-76a994f303a1/download/&dandisetId=001075&dandisetVersion=draft (variable depth series shows as TimeSeries not as ImageSeries)

E.g.: https://neurosift.app/?p=/nwb&url=https://api.dandiarchive.org/api/assets/fc5fca29-4c8f-444b-bbc1-4b8cd369d4ab/download/&dandisetId=001075&dandisetVersion=draft (nothing in processing shows as having a plugin; though they can also be considered simple TimeSeries)

Guessing there is further work needed somewhere in the specific plugins?

[magland]

Let's focus on the first one first.

It seems that Neurosift uses the spec embedded into the file and that takes precedence over the hard-coded rules that you modified. For your first example, the inheritance in the spec is:

VariableDepthMicroscopySeries PlanarMicroscopySeries MicroscopySeries TimeSeries

Look in the SPECIFICATIONS tab to see that.

What should we do?

[CodyCBakerPhD]

We should label 'MicroscopySeries' as being associated with the 'ImagingSeries' viewer

(they do not inherit because there are many properties of an ImageSeries that should not apply to the MicroscopySeries)

But the main data contained is structured very similarly and is intended to be viewed the same way

[magland]

Okay, try that first link now. Here's a shortcut to the tab https://neurosift.app/?p=/nwb&url=https://api.dandiarchive.org/api/assets/e6263508-26a0-4db0-ac0a-76a994f303a1/download/&dandisetId=001075&dandisetVersion=draft&tab=neurodata-item:/acquisition/PumpProbeImagingGreen|VariableDepthMicroscopySeries

Sorry the controls aren't labeled - just trying to scrap something together ;)

For the second link, which object are you interested in specifically?

[CodyCBakerPhD]

Sorry the controls aren't labeled - just trying to scrap something together ;)

The existing One/TwoPhotonSeries (and internally written ImageSeries) viewer such as (example) is perfectly fine - the central data storage is designed in nearly the same way so it should work - no need to create something entirely new just for this

Though from a UX standpoint I do like that button for incrementing individual frame indices

For the second link, which object are you interested in specifically?

For the MicroscopyResponseSeries, just like a standard time series (first dimension frames, second dimension ROIs) such as (example)

It's just like a normal RoiResponseSeries but it needed to be renamed to link to the new metadata objects for imaging spaces

Ditto for MicroscopySegmentations, they are just like ImageSegmentation containers of PlaneSegmentations, so the usual visual rendering of pixel/voxel masks is all that needs to be enabled for those

[magland]

Got it. Okay the first link should be working properly now.

For the MicroscopyResponseSeries and MicroscopySegmentations, I keep getting confused about the container relationships and what should correspond to what. If you go to the RAW view you can pull up the MicroscopyResponseSeries as a timeseries. But I guess we need that to be accessible from the NWB tab.

Maybe we could find time to have a quick call to figure out how it should look?

[CodyCBakerPhD]

For the MicroscopyResponseSeries and MicroscopySegmentations, I keep getting confused about the container relationships and what should correspond to what. If you go to the RAW view you can pull up the MicroscopyResponseSeries as a timeseries. But I guess we need that to be accessible from the NWB tab.

Yes, the raw view is exactly what do expose to the NWB tab, those look fine to me (I mostly want the ability then to pull up two sided by side views for the green/red signals, something I only know how to do from the NWB tab)

The container relationships are hopefully even more straightforward to explain than in core NWB:

'MicroscopyResponseSeriesContainercontains potentially manyMicroscopyResponseSeries`. The fact that I've made separate containers for each optic channel is arbitrary. The fact that I only have one series per container is particular to this dataset. Other datasets in the future could have multiple series stored per container, or might even combine signals across optic channels into a single container

likewise 'MicroscopySegmentations' is a container of 'MicroscopyPlaneSegmentations', which are just like the normal NWB PlaneSegmentations (though their container is referred to a 'ImageSegmentation' I believe in core NWB). Again, the fact this dataset split MicroscopySegmentations per channel and imaging type (with only one plane segmentation per container) is arbitrary, I could have also combined all three plane segmentations into a single container

[CodyCBakerPhD]

Got it. Okay the first link should be working properly now.

First link looks great! A minor wishlist request would be the ability to manually specify the 'video playback' speed (the slowest, 0.1x, maybe still goes too fast for microscopy data) and/or preload future chunks of the video to try to smooth the frame-by-frame rendering (this dataset should have been chunked pretty decently in this regard)

[magland]

@CodyCBakerPhD the containers should be working now, but MicroscopyPlaneSegmentation just shows up as a table. Are you expecting more?

[CodyCBakerPhD]

the containers should be working now

ROI responses look great! Thank you

but MicroscopyPlaneSegmentation just shows up as a table. Are you expecting more?

Yes, while a table is useful in its own right, the colorful ROI view like https://neurosift.app/?p=/nwb&url=https://api.dandiarchive.org/api/assets/b9807c25-b30e-438c-9c3d-782b65131cf9/download/&dandisetId=000728&dandisetVersion=0.240827.1809&tab=neurodata-item:/processing/ophys/ImageSegmentation|ImageSegmentation is preferred

[magland]

While the widget is there, it's a bit tricky to get the plumbing to all line up. I'm going to hold off on this for now and maybe revisit in a couple of weeks if you remind me.