Zarr v3

[normanrz]

This PR proposes to adopt the version 3 of the Zarr format for OME-Zarr.

Main changes:

• Only Zarr v3 is allowed for upcoming versions of OME-Zarr.
• Zarr v3 stores its metadata in zarr.json files. This is the same for arrays and groups. Attributes are now stored in a attributes object within the zarr.json files.
• The chunk_key_encoding of the Zarr arrays are respected instead of mandating the / dimension separator. Basically all Zarr features are available in OME-Zarr as they get approved through the ZEP process.
• Add a ome namespace within the zarr.json attributes.
• Move version fields from multiscale, plate, well etc. up into the ome object.
• Move the registration of labels from the labels group into the multiscale group.
• Rename the document to OME-Zarr specification

This PR is currently in a draft status and builds upon the #138 PR by @bogovicj. Check this link for a more convenient review: https://github.com/ome/ngff/pull/206/files/b92f540dc95440f2d6b7012185b09c2b862aa744..HEAD

[github-actions[bot]]

Automated Review URLs

• Readthedocs
• render latest/index.bs
• diff latest modified

[d-v-b]

• Add a ome namespace within the zarr.json attributes.

• Move version fields from multiscale, plate, well etc. up into the ome object.

Thank you for this!

[imagesc-bot]

This pull request has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/adopt-zarr-v3-in-ome-zarr/84786/1

[joshmoore]

After a quick read, @normanrz, here are my initial thoughts (where not simply a big :+1:):

Add a ome namespace within the zarr.json attributes. Move version fields from multiscale, plate, well etc. up into the ome object.

These two together make me wonder whether or not the key couldn't be something more sophisticated, like the version itself:

"https://ngff.openmicroscopy.org/0.5": {
   ...
}

It's somewhat non standard to use a more complicated key like that, but it's a value that we will have in memory anyway to load the schema files, etc.

Move the registration of labels from the labels group into the multiscale group.

The labels "object" was definitely poorly conceived originally and so improvements welcome, but I'll note there are a handful of open issues as well as integration with the upcoming SpatialData work that might be worth considering. If it becomes more of an issue, I'd suggest splitting this out of your proposal here (if possible).

Rename the document to OME-Zarr specification

I know this is a constant source of confusion/frustration, but I think we might want to reconsider this.

[normanrz]

Add a ome namespace within the zarr.json attributes. Move version fields from multiscale, plate, well etc. up into the ome object.

These two together make me wonder whether or not the key couldn't be something more sophisticated, like the version itself:

"https://ngff.openmicroscopy.org/0.5": {
   ...
}

It's somewhat non standard to use a more complicated key like that, but it's a value that we will have in memory anyway to load the schema files, etc.

Fine with me. It is a bit more verbose, but we could get rid of the additional "version" attribute.

Move the registration of labels from the labels group into the multiscale group.

The labels "object" was definitely poorly conceived originally and so improvements welcome, but I'll note there are a handful of open issues as well as integration with the upcoming SpatialData work that might be worth considering. If it becomes more of an issue, I'd suggest splitting this out of your proposal here (if possible).

Also, fine with me to split it out. But I really think label images in OME-Zarr need to be reworked.

Rename the document to OME-Zarr specification

I know this is a constant source of confusion/frustration, but I think we might want to reconsider this.

That is certainly up for debate. But my preference would be to go with OME-Zarr as title of the document, because that is what it is about.

[normanrz]

Closing this in favor of #227. Discussion will continue there.