search

ome / ome2024-ngff-challenge

Project planning and material repository for the 2024 challenge to generate 1 PB of OME-Zarr data
https://pypi.org/project/ome2024-ngff-challenge/
BSD 3-Clause "New" or "Revised" License
11 stars 8 forks source link

How to represent lightsheet data with multiple timepoints / tiles / channels #45

Closed constantinpape closed 1 week ago

constantinpape commented 2 weeks ago

For the challenge we want to convert lightsheet data that can have multiple timepoints, tiles and channels. It does not make sense to stack all of this across a single array (and might not even be possible since dimensions might change across different timepoints). So each data needs to contain multiple arrays (with a multiscales each), which I can convert to zarr v3 thanks to the conversion tool.

The question is how the high level data organisation should be. For this data we currently use the n5 data format defined by BigDataViewer (https://github.com/bigdataviewer/bigdataviewer-core/blob/master/BDV%20N5%20format.md). In this format each multiscale array is stored as setupX/timepointX, where the setup can encode different attributes (e.g. channel, illumination, tile).

How would we best translate this to ome zarr?

joshmoore commented 2 weeks ago

Hi @constantinpape,

Without an NGFF spec, I don't have any definitive help. But since the challenge isn't formally versioned, I'd suggest converting each of the multiscales separately and they put as complete a representation of the setup metadata (and/or anything else needed) into the zarr.json. Hopefully that process could start to inform a future spec. It might be worth additionally adding something as close to an existing spec (e.g., bioformats2raw.layout?) as possible so existing tools can at least discover the multiscales. They wouldn't understand the BDV representation yet, but that's something we can work towards.

~J.

constantinpape commented 1 week ago

Hi @joshmoore , I decided to just translate the layout of BDV. I have only very limited time to work on this and can't investigate other specs.