Add ome-zarr support

[thewtex]

Current state of the spec is here, @joshmoore?:

https://github.com/ome/omero-ms-zarr/blob/master/spec.md

[joshmoore]

Currently, yes. I'll post something when it moves.

[thewtex]

@joshmoore thanks!

[oeway]

Supporting ome-zarr or have a general way to interact with remote multi-resolution data would be awesome.

I had some discussion with others about potential way to display huge volume with itk-vtk-viewer from Python with imjoy-rpc, one potential solution is to support precomputed pyramid with zarr (as implemented in vizarr, but also for 3D).

Looking forward to this feature.

[joshmoore]

The spec has now moved to https://github.com/ome/ngff (rendered at: https://ngff.openmicroscopy.org/latest/)

[joshmoore]

Hi @thewtex. We were looking to start exploring the viewer for opening lightsheet data in OME-Zarr and ran across https://github.com/Kitware/itk-vtk-viewer/pull/343 & https://github.com/Kitware/itk-vtk-viewer/issues/176. A few questions:

• which ticket/PR should we use for discussion? ;)
• can you give us a status update?
• is there anything you could use help with?
• if yes, can you provide some pointers for getting started with the code base?

Hope all is well. ~Josh

[thewtex]

Hi @joshmoore !

Thanks for the questions, and comments! Yes #343 comprises a major refactoring that relates to multi-scale supported; it will be merged at the end of this week. This will initially support multi-scale support for large in-browser-memory images. However, I will immediately begin work to address #176, zarr, client-side backend support. I will provide an initial example link, but your feedback on both the zarr support and the viewer features will be highly appreciated! The refactoring is aiming for compatibility with OME-Zarr and the ome-zarr label features. But, I am sure that your participation would improve both the standard compatibility and viewer features. Looking forward to working with you! Stay safe and well.

[oeway]

@thewtex Wondering what left to support ome-zarr format?

[thewtex]

We have to support the different dimension order / memory layout of ome-zarr (contiguous channels) in addition to the current support, (interleaved channels).

I will implement this after adding support for selection of a region of interest.

[thewtex]

Test suite data sets:

• https://github.com/ome/ome_zarr_test_suite/tree/main/data
• https://ome.github.io/www.openmicroscopy.org/2020/11/04/zarr-data.html
• https://github.com/ome/ome-zarr-py/tree/master/tests/data

[joshmoore]

@sbesson has also been working on the next batch of v0.4 data that will get pushed to EBI's S3. (As so often, largely blocked by naming discussions. :wink:) It is likely to contain at least some of @constantinpape's samples from https://github.com/ome/ome-ngff-prototypes

[PaulHax]

465 Starts support of 0.4

TODO: Add translate transform. Add multi channel support. Add test for datasets here: https://github.com/ome/ome-ngff-prototypes and https://github.com/ome/ome_zarr_test_suite/tree/main/data

[oeway]

Great work @PaulHax ! I am wondering whether I can use it already through the ImJoy API (i.e. the setImage function)? What's the format? Can I pass a zarr store? I would like to test it in jupyter lite so we can view large zarr files in the browser.

[PaulHax]

Thank you @oeway . setImage can take a URL object pointing to the store:

new URL(https://minio-dev.openmicroscopy.org/idr/v0.4/2022-02-03/idr0062/6001240.zarr)

The viewer then loads the entire least detailed image. If the frame rate is high enough, the viewer may continue fetching more detailed images.

[oeway]

@PaulHax Can we allow passing a store object (with getItem(key) and containsItem(key) functions) to setImage? The idea is that we will use the imjoy-rpc layer to proxy a store implemented in Python and pass it to setImage in itk-vtk-viewer.

[PaulHax]

Ah, yes. Passing a "Store" object with a getItem(key) function to setImage also works.

[oeway]

Great! Looks like the store object is detected and the viewer shows the control UI, however it gives me KeyError: '3/0undefined0undefined0undefined1undefined0undefine'.

Full browser log: jupyter.imjoy.io-1650558105187.log

Here is how to reproduce:

1. download itk-vtk-viewer-demo-ome-zarr.ipynb.zip and unzip the notebook
2. open https://jupyter.imjoy.io/lab/index.html drag and drop the notebook to the file tree located on the left side, then double click to open it.
3. Wait until you see an ImJoy icon in the toolbar, then execute cell by cell.
4. In the notebook, I created a small ome-zarr file, which has already been tested with vizarr, you can also run the last cell to see how it looks like in vizarr.
5. If you want to debug with your local instance of itk-vtk-viewer, replace "https://kitware.github.io/itk-vtk-viewer/app" into your local url.

[PaulHax]

@oeway Thank for the step by step and the new test case. Threw in a default "dimension_separator" in #472 to keep the errors at bay. Still need to do multi-channel though.

[oeway]

@PaulHax Great, thanks for the quick fix!

It works now!

Looking forward to the multi-channel support too.

[oeway]

@thewtex BTW, any timeline for the multi-channel support? Really excited about the progress here!

[thewtex]

@oeway love to see the imjoy/jupyterlite demo!

multi-channel support is expected within the next two weeks.

[oeway]

@oeway love to see the imjoy/jupyterlite demo!

multi-channel support is expected within the next two weeks.

Sounds great! Looking forward.

BTW, I just tried the jupyterlite demo again, and it seems the latest itk-vtk-viewer doesn't show the zarr image anymore.

To reproduce, the ImJoy flavor of jupyterlite now support loading notebooks from URL, so one can click a like like this to see the itk-vtk-viewer NGFF demo: https://jupyter.imjoy.io/lab/index.html?load=https://gist.github.com/oeway/4a3ec3866eae199f982b8b81bf72a3a8&open=1

@PaulHax @thewtex

[thewtex]

Hi @oeway ,

I tried, and seemed to get the same output (without the channel support),

The issue I ran into was Run all did not produce the output -- I had to run the cell individually. Is this the behavior you are seeing, or something else?

[oeway]

Hi @thewtex Thanks for testing it. I did run individually, however, I still don't see the image:

I am using Chrome Version 100.0.4896.127 (Official Build) (x86_64)

[thewtex]

@oeway v12.3.0 incorporates #480 , interested to know if you still have this issue?

[oeway]

@oeway v12.3.0 incorporates #480 , interested to know if you still have this issue?

Thank you, after reloading, I get the latest version! However, there seems to be an issue with the display:

Try it here with Chrome: https://jupyter.imjoy.io/lab/index.html?load=https://gist.github.com/oeway/4a3ec3866eae199f982b8b81bf72a3a8&open=1

[thewtex]

@oeway nice, easily reproducible example :-D

It worked for me:

But it is dim -- this will be addressed here: https://github.com/Kitware/itk-vtk-viewer/pull/488

[oeway]

Hey, @thewtex It's working! Now it's actually comes nicely together! (It seems still dim though, is the fix deployed?)

I am now using zarr+ZipStore to access a remote zipped NGFF file served from an S3 server (with the possibility to add authentication using presigned URL), completely in Pyodide/JupyterLite.

Try it here: https://jupyter.imjoy.io/lab/index.html?load=https://gist.github.com/oeway/391b4352ea57b5682366ce3dc2fa9174&open=1

[oeway]

But it is dim -- this will be addressed here: https://github.com/Kitware/itk-vtk-viewer/pull/488

I just checked and it looks still dim, could you take another look? @thewtex

[thewtex]

@oeway yes, that is quite annoying. The #488 corrected the Window/Level locally (same behavior was observed), but I also still see the issue with a 2D Zarr. Passing the issue to @PaulHax :gift:

[thewtex]

With #499 , released in 13.0.0, the 2D multi-component rendering is now addressed! :tada: :taco: h/t @PaulHax

Verified by clearing the cache with your example @oeway :+1:

We now have a good test suite demonstrating support for a number of OME-NGFF-Zarr images, so closing. If new issues are discovered, let's create new issues to start fresh threads. Thanks!

[joshmoore]

:tada: Looking forward to passing out the likes of https://kitware.github.io/itk-vtk-viewer/app/?image=https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.1/6001241.zarr&rotate=false to people :smile:

[oeway]

FYI: @thewtex the following link does not give me the but only 2 channels in the UI:

https://kitware.github.io/itk-vtk-viewer/app/?image=https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.1/6001241.zarr&rotate=false

Oh, wait, it's working after waiting for a while, never mind.