scverse / spatialdata

An open and interoperable data framework for spatial omics data
https://spatialdata.scverse.org/
BSD 3-Clause "New" or "Revised" License
174 stars 34 forks source link

Incremental io #525

Open ArneDefauw opened 1 month ago

ArneDefauw commented 1 month ago

PR for illustration of possible workflow for adding elements to spatialdata object inspired by https://github.com/scverse/spatialdata/blob/582622f689a7e05421e9d066f98baf702549978f/tests/io/test_readwrite.py#L136 and related to discussion https://github.com/scverse/spatialdata/discussions/520.

Currently only implemented for image and labels, but similar approaches would work for points, shapes and tables.

See unit test for example usage. https://github.com/ArneDefauw/spatialdata/blob/2e71e63ca9ccbd849879efe1f26c7221cdd60b34/tests/io/test_readwrite.py#L662

Approach also allows for writing to multiscale, and avoids recomputation via writing to intermediate slot.

Some things are not covered yet, such as automatic reload, and proper passing of transformations to spatialdata.models.Image2Dmodel.parse

I am still unsure to what extent spatialdata should implement such workflows (e.g. overwriting elements on disk), or should merely give suggestions about possible workflows (via notebooks/documentation)