support for z-stacks

[hspitzer]

Type of the feature

• [x] New function in sqduipy.im?
• [ ] New function in squidpy.gr?
• [x] New function in squidpy.pl?
• [ ] Change an existing functionality, such as default behavior?
• [ ] Other?

Description

We should support datasets with a z-dimension properly. Broadly, the following functions are needed:

• infrastructure:
  • [ ] imagecontainer adaption to z-stacks
  • [ ] adata adaption to z-stacks
• analysis methods:
  • [ ] graph statistics with 3D coords
• visualisation:
  • [ ] of results with 3D coords
  • [ ] interactive visualisation; allowing to switch between different z-coords

Below a couple of questions that I have regarding the infrastructure and visualisation parts:

How to store z dimension in adata and in img?

• adata: add column to adata.obsm['spatial'] such that it stores x, y, z
• img:
  • (a) add z coord to DataArray such that we store all images in the same DataArray.
    • makes processing single images quite complicated as soon as we have more than one z
  • (b) add z coord as attribute to each DataArray
    • know immediately which z this belongs to, but need to make sure to always track this attribute.

How to modify interactive viewer to deal with z

• (a) limit to 1 z coord on initialisation (both imagecontainer and adata)
• (b) prefix point layers with z coord to enable multiple point layers (could visualise layers that have different z coords on top of each other)
  • might be confusing, how can the user know which layers actually belong to each other?
• (c) have multiple z coords, but only allow matching layers to be displayed (img + genes)
  • requires tight coupling between layers that is not present atm

related: #309

happy to discuss @giovp and @michalk8

[michalk8]

What do we expect to store in the Z-dim? Is it large like X/Y or more like additional image slices?

imagecontainer adaption to z-stacks

Right now, simplest would be to assume that Z axis is always present, 2D would be just a special case. Same for adata - we can always add some constant (e.g. 1s) for the 3rd dimension.

(a) limit to 1 z coord on initialisation (both imagecontainer and adata)

Would not go for this one - it's easy, but very limiting. Maybe some slider could be used, inspiration: https://napari.org/tutorials/fundamentals/image.html#d-rendering-of-images

[hspitzer]

What do we expect to store in the Z-dim? Is it large like X/Y or more like additional image slices?

I think for now we can assume a small z-dim, so like additional image slices.

Right now, simplest would be to assume that Z axis is always present, 2D would be just a special case. Same for adata - we can always add some constant (e.g. 1s) for the 3rd dimension.

Yes, lets do that. I think wrt ImageContainer representation (a) (DataArray with x,y,z,channels) is cleanest. We could add an attribute library_id to the DataArray, tracking for the name of the library_id for each z. Then the processing functions could have a library_id argument that indicates which z slice is being processed.

To create a ImageContainer with more than one z I'd start with just having a concat function that takes a list of ImageContainers, z_coords, and library_ids and creates a joint ImageContainer.

(a) limit to 1 z coord on initialisation (both imagecontainer and adata)

Would not go for this one - it's easy, but very limiting. Maybe some slider could be used, inspiration: https://napari.org/tutorials/fundamentals/image.html#d-rendering-of-images

yes, you are right, a slider is the way to go. The way I imagine this sliding through z would internally slice the image and adata (points layers) that are shown.

Ok, I'll create a PR with how I imagine ImageContainer to work in the next days. I might need some help with the napari slider though.

[michalk8]

I think for now we can assume a small z-dim, so like additional image slices.

Ok, I see, thanks!

Yes, lets do that. I think wrt ImageContainer representation (a) (DataArray with x,y,z,channels) is cleanest. We could add an attribute library_id to the DataArray, tracking for the name of the library_id for each z. Then the processing functions could have a library_id argument that indicates which z slice is being processed.

Sounds good to me.

Re visualization: I will take care of napari, should not be an issue. We should also take inspiration from giotto's plots (3D scatterplot or its slices), see https://rubd.github.io/Giotto_site/articles/mouse_starmap_cortex_200917.html

[michalk8]

closed via #329