Design of WSI data structure

[Mr-Milk]

The current design of the data structure to represent WSI contains several drawbacks. This issue is to discuss if there are better solutions or if the current one is already acceptable.

Another thing to discuss here is if we should ship the WSIData as a separated package, think of anndata and scanpy.

Design

We use a WSIData class that abstracts two things

1. Read WSI
2. Read/Write for SpatialData

Image

• The WSIData interacts with WSI using the reader. However, the WSI can load as a DataTree which is the pyramids structure in xarray and it's lazy, but this will raise the problem when saving the SpatialData on disk, the DataTree will need to make a copy of the uncompressed slide data, this could result in crazy disk space usage. Currently, the WSIData will not create a DataTree representation of the slide file by default.
• wsi_thumbnails which is small on disk and memory is attached to the SpatialData object when reading the slide for the first time.

Pros

1. Easy implementation
2. Performant IO if interacts with slide image
3. Avoid saving a copy of the slide

Cons

1. User needs to keep a paired slide file and SpatialData file
2. Cannot fully integrate with the SpatialData ecosystem

Tissue contours

Attach in shapes slot, the tissue holes should be passed to holes parameters when constructing Polygon.

Tiles

Attach in shapes slot

Tiles are saved as polygon, the tiles table should also record the x and y in image coordinates at level 0.

A tile_spec should be recorded.

Feature Matrix

This is always a 2D array, attach in images slot

Tile neighbors sparse matrix

Save in the obsp slot of an AnnData object in the tables slot of SpatialData

[afrendeiro]

Thanks for writing this up.

A few initial thoughts:

• Separating WSIData from LazySlide makes sense, especially if there is considerable engineering on the data side.
• I'm not sure "User needs to keep a paired slide file and SpatialData file" is really a downside. The slide file is something one wants to keep immutable, and is anyway in most cases a proprietary format that I'm not too keen on interfacing with beyond reading.
• I don't think I'm familiar enough with SpatialData to really see where is the bottleneck. Why would this solution "[..]not fully integrate with the SpatialData ecosystem"? Is it 'just' the lack of a greedy DataTree made from the image data by default? If so, it doesn't seem too concerning.
• I think you mentioned previously, but now I'm not sure why feature matrices are saved in the labels slot. The docs say values can only be integers. What's the drawback of using tables?
• Why using SpatialData would indeed be preferable, we shouldn't forget to write a few .to_{x} functions, to quickly convert data to e.g. torch geometric data. I'm curious whether storing neighours graphs in obsp is performant to the same degree as everything natively in torch geometric format.

[Mr-Milk]

Thanks for sharing your thoughts, I agree with most of the points.

• You are right about the labels, the reason why I don't use tables is because tables slot is still not lazy at the current stage, if the feature matrix is big, it will take a long time to initiate a data object. I'm considering images slot for easy accessing the data.
• As for datasets, I would prefer using functions to convert the WSIData into datasets instead of implementing class methods, this way we will have more extensibility for future unseen use cases. I'm considering adopting the huggingface datasets, which can be easily adapted to different deep-learning frameworks (tf, torch, jax, ...) by calling the .with_format('tf') for example.
• For the neighboring matrix, I'm trying to align with the scverse standard for better interoperability, so the user can easily apply tools like scanpy and squidpy (eg. for spatial domain detection). There will be two sparse matrices, connectivities and distances matrices. For the neighboring matrices, we can easily construct graph datasets for the user to support different deep-learning frameworks.

[Mr-Milk]

However, if we think feature matrix for WSI as gene expression in spatial omics. It actually make more sense to save them in tables.