Closed constantinpape closed 2 years ago
I started to work on this:
I have created an example project with "bdv.ome.zarr" and "bdv.ome.zarr.s3" formats. This means the data is in ome.zarr format, but we are still using a bdv xml file in MoBIE, because we can't save the voxel size in ome.zarr yet (for v0.3). The example project is available:
/g/kreshuk/pape/Work/mobie/covid-em-datasets/ngff-example/data
The available image data (ome.zarr and xml files) can be found in the folder Covid19-S4-Area2/images/bdv.ome.zarr
.
So we can already start with adapting the readers in MoBIE:
Goal: the fiji command to open ome.zarr from the filesystem should work with the raw.ome.zarr
file.
What needs to be adapted for the ome.zarr v3 format (list of necessary changes may not be exhaustive):
transform
field does not exist, but the current code should still work, because it uses try-excepts
, e.g here and here.axes
field contains
["z", "y", "x"]
, i.e. is 3 dimensional. And we need to get rid of other 5d changes, e.g. here.Goal: the fiji command to open ome.zarr from s3 should work for loading the raw.ome.zarr
file from s3.
This should not require any additional changes once the filesystem version works.
Once https://github.com/mobie/mobie-viewer-fiji/pull/293 is merged, we can add the bdv.ome.zarr
and bdv.ome.zarr.s3
file formats and integrate the loaders with the mobie spec, so that the example project can be loaded in MoBIE (both from the filesystem and from s3).
The PR https://github.com/mobie/mobie-viewer-fiji/pull/297 is progressing and I think we should have a first version that can read the example data tomorrow or early next week.
Once that works and is cleaned up a bit we should make a larger clean up and extension of the (ome) zarr readers:
dimensionSeparator
from the zarr .attrs instead of hard-coding it. (Using "." as default if it's not there.)axes=["y", "x"], ["c", "y", "x"], ["c", "y", "x"], ["t", "c", "y", "x"], ["t", "c", "z", "y", "x"]
and for dimensionSeparator="."
. (I will do this.)I have generated ngff example data for all the different axes combinations now. It's available on:
/g/kreshuk/data/ngff/v.03
https://s3.embl.de/i2k-2020/ngff-example-data/v0.3
I have now also added a project that uses the "ome.zarr" data format (in contrast to "bdv.ome.zarr" this does not have an additional xml and all image metadata is read directly from the zarr file): https://github.com/mobie/covid-if-project.
The project with all data is here: /g/kreshuk/pape/Work/mobie/covid-if-project
(I haven't uploaded it to s3 yet and it's to large to zip and upload to owncloud).
@tischi, I just discussed how to continue here with @KateMoreva:
I can open the project with the current master, using this:
new MoBIE("https://s3.embl.de/i2k-2020/project-bdv-ome-zarr", MoBIESettings.settings().imageDataFormat(ImageDataFormat.OmeZarr));
Is there a reason why this is not called ImageDataFormat.BdvOmeZarrN5
?
Is there a reason why this is not called
ImageDataFormat.BdvOmeZarrN5
?
I think it should be called ImageDataFormat.BdvOmeZarr
.
The fact that we use n5 for opening is an implementation detail, that should not be part of the format name.
Sorry, I meant: ImageDataFormat.BdvOmeZarrS3
I can open the project with the current master, using this:
new MoBIE("https://s3.embl.de/i2k-2020/project-bdv-ome-zarr", MoBIESettings.settings().imageDataFormat(ImageDataFormat.OmeZarr));
Is there a reason why this is not called
ImageDataFormat.BdvOmeZarrN5
?
Yes, it should be named BdvOmeZarr once MoBIE could open ome.zarr files (not bdv.ome.zarr). I can push renamed files and some more changes right now if you wish (but so far MoBIE can open only BbvOmeZarr, thus the name was just OmeZarr)
Quick summary from a meeting with @KateMoreva and @tischi:
the progress on this is great and we can read all current flavors of ome.zarr now (except for from ome.zarr.s3, but only because this is missing an example project, which I will provide). The next step will be to continue refactoring this functionality into the two separate "bigdataviewer-n5" and "bigdataviewer-ome-zarr" repositories:
The next steps for this are (not necessarily in this order):
@KateMoreva I have created the example project for ome.zarr.s3 now:
https://s3.embl.de/i2k-2020/project-ome-zarr
.
Add "prototype" support for ome.zarr fle format:
I will work on the first 3 points, @KateMoreva will work on implementing the reader in MoBIE (once I have created the example project!). The last point would probably be done by @K-Meech, but I am not sure if we need to work on this already, I just added it here for completeness.
cc @joshmoore: this will also serve as example data and reader for v0.3.