Unable to load image mask

[CodyCBakerPhD]

https://neurosift.app/?p=/nwb&url=https://api.dandiarchive.org/api/assets/f02db27e-82eb-41dd-865a-a08bb41491da/download/&dandisetId=000728&dandisetVersion=draft&tab=neurodata-item:/processing/ophys/ImageSegmentation|ImageSegmentation

index-d963a21d.js:228 Uncaught (in promise) Error: Unable to load image mask dataset
    at bxA.initialize (index-d963a21d.js:228:8805)
    at async index-d963a21d.js:228:9854

[CodyCBakerPhD]

[magland]

This is because pixel_map is not supported -- only image_mask.

I'm putting this on my todo list.

[CodyCBakerPhD]

Ah, I see

Hopefully shouldn't be too bad

Might as well include voxel_mask in that too, example: https://neurosift.app/?p=/nwb&url=https://api.dandiarchive.org/api/assets/ac3cd1e2-ed6e-49c5-9ab6-281bebed78e6/download/&dandisetId=000350&dandisetVersion=0.221219.1506

[bendichter]

@magland , would it be possible to address this?

This is going to be a more prominent error when we release and advertise the Allen Institute Visual Coding Ophys dataset

[magland]

@bendichter @CodyCBakerPhD I almost have this working. However, in the case of pixel_mask the nwb file does not seem to have the information of the width/height of the image. For that it seems I will need to look in other parts of the file such as SummaryImages. Could you poke around the data in the Raw tab and see if you have advice for this?

[magland]

I have this working for the pixel_mask case (your first example link @CodyCBakerPhD ) but not for voxel_mask.

As mentioned in my previous comment, the component does some ad hoc fishing around the parent group of the nwb file to try to determine the overall image shape. Right now it looks for an "Image" dataset inside an "Images" group in the parent group. It works for that particular example.

[CodyCBakerPhD]

I believe there's an issue on the schema to add a record of max image size when pixel/voxel mask is used, for exactly this type of situation

Right now it looks for an "Image" dataset inside an "Images" group in the parent group. It works for that particular example.

That may not always work because there might be separate images from different devices at different resolutions - the safest way might just be to 'infer' the best guess of boundary based on the min/max of the pixels/voxels?

Will bring this up again at next LBNL meeting to try to push a proper way of specifying the information in the object

[bendichter]

The maximum_intensity_projection summary image looks like it is vertically flipped from the PlaneSegmentation image. I'm not sure whether this is a visualization or the way this data was saved.

[CodyCBakerPhD]

The maximum_intensity_projection summary image looks like it is vertically flipped from the PlaneSegmentation image. I'm not sure whether this is a visualization or the way this data was saved.

Flipping control was requested in https://github.com/flatironinstitute/neurosift/issues/155

Likewise, flipping control would be nice for masks as well - towards the purposes of aligning all 3 with frames from the raw imaging, for instance

[magland]

I'd like to avoid a flipping control in the case where there is one clear orientation.

I have fixed this in this particular case and I am guessing that this will now be correct for other examples. We'll see if this arises again indicating inconsistency in how the data are stored in nwb.

[CodyCBakerPhD]

I'd like to avoid a flipping control in the case where there is one clear orientation.

There is not one clear orientation most of the time for ophys - that would only be for visual stimuli presented to a subject - for images of neural data or even natural behavior (mazes, etc) there may not be a clear up or down and experimenters often re-orient on the fly to better visualize different patterns

Flipping control on all images is the most surefire way to resolve this across the board, and is useful as a general way of comparing different views

[magland]

Okay I have added a plane transformation control to the PlaneSegmentationView.

However I have not properly implemented one case (xy-flip for the image_mask). I feel like my brain is too old to figure this out:

https://github.com/magland/fi-sci/blob/e0f600fb3c491e6c756dd79f30faa0f802958a6c/apps/neurosift/src/app/pages/NwbPage/viewPlugins/ImageSegmentation/PlaneSegmentationView.tsx#L217-L248

But all the other cases work.

pixel_mask:

https://neurosift.app/?p=/nwb&url=https://api.dandiarchive.org/api/assets/f02db27e-82eb-41dd-865a-a08bb41491da/download/&dandisetId=000728&dandisetVersion=draft&tab=neurodata-item:/processing/ophys/ImageSegmentation%7CImageSegmentation

image_mask:

https://neurosift.app/?p=/nwb&url=https://dandiarchive.s3.amazonaws.com/blobs/368/fa7/368fa71e-4c93-4f7e-af15-06776ca07f34&tab=neurodata-item:/processing/ophys/ImageSegmentation%7CImageSegmentation