search

AllenCellModeling / napari-aicsimageio

Multiple file format reading directly into napari using pure Python
GNU General Public License v3.0
33 stars 9 forks source link

feature/store-access-to-aicsimage-and-raw-meta-in-layer-meta #43

Closed evamaxfield closed 2 years ago

evamaxfield commented 2 years ago

Pull request recommendations:

https://forum.image.sc/t/napari-file-open-and-metadata-access/63497/12

Thanks for contributing!

Can't tag as reviewers but would love opinions from @tlambert03, @psobolewskiPhD, and @pr4deepr

evamaxfield commented 2 years ago

In addition to the screenshot in the readme, might be nice to add a bulleted list of specific keys and their meanings aicsimage, ome_types and metadata (or is it raw? screenshot and code seem to conflict)

Agree. I will document!

It's the layer["metadata"] that has a Dict of extra stuff

so its layer["metadata"]["raw"] for the "raw metadata" and layer["metadata"]["ome_types"] for the ome object

psobolewskiPhD commented 2 years ago

This is a bit over my head, but I think it works, according to your OP? This is for a LIF, so getting the Element

viewer.layers[0].metadata
Out[1]: 
{'aicsimage': <AICSImage [Reader: LifReader, Image-is-in-Memory: False]>,
 'raw': <Element 'LMSDataContainerHeader' at 0x14d5a67c0>}

viewer.layers[0].metadata["aicsimage"]
Out[3]: <AICSImage [Reader: LifReader, Image-is-in-Memory: False]>

viewer.layers[0].metadata["raw"]
Out[4]: <Element 'LMSDataContainerHeader' at 0x14d5a67c0>
pr4deepr commented 2 years ago

Thanks for making this @JacksonMaxfield The aicsimage object access works well for me!

  • Adds both "raw" and optionally "ome" metadata in the metadata dict -- @tlambert03 currently this change means that users reading anything besides OME-TIFF and CZIs will not have a metadata viewer. Any suggestions on how to create a new "OME" metadata render tree with a different name for the element tree meta?

I think a different name for the element tree meta, such as xml or as you use now, raw should work. If you get an xml for the metadata, can you convert it into an ome tree using ome_types library written by @tlambert03 ?

pr4deepr commented 2 years ago

Oh yes, when I read a czi file I get this:

image

evamaxfield commented 2 years ago

If you get an xml for the metadata, can you convert it into an ome tree using ome_types library written by @tlambert03 ?

This how it works in the current version. OME Types will render the ElementTree.... I thoughts that why you were confused?

This change will make it only render the tree if it has a metadata converter to OME OR it is an OME file. BUT now the metadata is accessible in the console -- unless we come up with a method to spawn two tree renderers.

If you are asking if we can convert the raw metadata into OME thats the whole deal of like "needing a converter" -- CZI has a converter which is XSLT, bioformatsReader has a converter which is bioformats, but not all readers have converters.

And yes @psobolewskiPhD that is exactly correct.

psobolewskiPhD commented 2 years ago

Oh snap! I think I get it now!

viewer.layers[0].metadata["aicsimage"].scenes
Out[4]: ('A3 Region2_Merged', 'A3 Region1_Merged')
viewer.layers[0].metadata["aicsimage"].dims.order
Out[5]: 'TCZYX'

This is pretty sick! Very cool—the layer data is actually squeezed of those extra dims, so having this info here is quite cool. :heart:

evamaxfield commented 2 years ago

Seems like I definitely need more documentation in the README as to "how this works"...

Taking suggestions but how does this sound:

To access the AICSImage object used to load the image data and metadata, you can do the following in the console:

img = viewer.layers[0].metadata["aicsimage"]

This is a normal AICSImage object so all the normal operations apply:

img.dims.order  # TCZYX
img.channel_names  # ["Bright", "Struct", "Nuc", "Memb"]
img.get_image_data("ZYX")  # np.ndarray

The metadata dict also stores shorthands to two variants of the metadata:

viewer.layers[0].metadata["raw"]

Stores the metadata in whichever format is used by the underlying file format reader, i.e. for CZI it returns an xml.ElementTree, for OME-TIFF it returns an OME object from ome-types.

Lastly, if the underlying file format reader has an OME metadata converter, you may additionally see a key in the metadata dict called "ome_types". For example, because the aicsimageio CZIReader and BioformatsReader both support converting metadata to OME metadata, you will see an "ome_types" key.

viewer.layers[0].metadata["ome_types"]  # OME object from ome-types

?

Again, I want to emphasize, that this PR currently removes the ability to render the ElementTree in the napari viewer. ome-types napari plugin specifically looks at the "ome_types" metadata key for metadata pickup, it just so happened that it allowed rendering of an ElementTree in addition to an OME object, and so we were overloading the functionality there. If this PR gets merged as is, LIF files for example wont have a metadata tree render because the ElementTree metadata has been moved from "ome-types" to "raw".

pr4deepr commented 2 years ago

This how it works in the current version. OME Types will render the ElementTree.... I thoughts that why you were confused?

Thanks for clarifying. Yea, I wasn't clear on how the current implementation works, only that it gets an ElementTree.

Actually, I just realised. I am not getting an ome_types key for my czi file. I only get the raw key. I think its got to do with the czi file being from the Zeiss lattice microscope which has different metadata compared to traditional czi files.

When I tried another CZI file, I get the following: image

pr4deepr commented 2 years ago

Small suggestion for the documentation, perhaps preface with all the changes (if you haven't already):

The metadata dict now currently stores a reference to the AICSImage object used to load the image data and image metadata.

evamaxfield commented 2 years ago

Actually, I just realised. I am not getting an ome_types key for my czi file. I only get the raw key. I think its got to do with the czi file being from the Zeiss lattice microscope which has different metadata compared to traditional czi files.

Yep. and for that file you also don't get the render anymore... Is that okay? If you want to fix that btw you can make a PR to: https://github.com/AllenCellModeling/czi-to-ome-xslt

pr4deepr commented 2 years ago

Thanks @JacksonMaxfield . Its not a big issue now. I'll make a PR at czi-to-ome-xslt as you mentioned. Appreciate the prompt responses!

evamaxfield commented 2 years ago

Alright, README is updated. Hope the new language clears up any confusion.

Before I merge this, I want to double check: Everyone is okay with me basically removing the metadata tree renderer for any file that isn't itself an OME file or doesn't have an OME metadata transformer? LIF metadata would only be accessible via terminal, ND2 metadata would only be accessible via terminal, etc.

codecov-commenter commented 2 years ago

Codecov Report

Merging #43 (86e126f) into main (3fce01f) will increase coverage by 0.54%. The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main      #43      +/-   ##
==========================================
+ Coverage   83.05%   83.59%   +0.54%     
==========================================
  Files           5        7       +2     
  Lines         177      189      +12     
==========================================
+ Hits          147      158      +11     
- Misses         30       31       +1
Impacted Files Coverage Δ
napari_aicsimageio/core.py 86.08% <100.00%> (+0.63%) :arrow_up:
napari_aicsimageio/tests/__init__.py 100.00% <0.00%> (ø)
napari_aicsimageio/__init__.py 83.33% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 3fce01f...86e126f. Read the comment docs.

tlambert03 commented 2 years ago

Before I merge this, I want to double check: Everyone is okay with me basically removing the metadata tree renderer for any file that isn't itself an OME file or doesn't have an OME metadata transformer? LIF metadata would only be accessible via terminal, ND2 metadata would only be accessible via terminal, etc.

I'm fine with this, and also, if it's possible, I would be happy to update the ome-types tree renderer to look for these keys if there's a clear way they can be shown?
was LIF or ND2 working anyway with the OME tree viewer? is there anything I can do to make them work?

evamaxfield commented 2 years ago

I'm fine with this, and also, if it's possible, I would be happy to update the ome-types tree renderer to look for these keys if there's a clear way they can be shown? was LIF or ND2 working anyway with the OME tree viewer? is there anything I can do to make them work?

Oh weird... I just installed the latest version napari-aicsimageio and I guess I was just completely mistaken I don't see the metadata render on LIF or CZI :woman_shrugging:. Now I am confused but I guess this PR can be merged now without any complaint from me :joy:

evamaxfield commented 2 years ago

That said, I am going to update the "raw" metadata dict key to "raw_image_metadata" so its a bit more clear what is stored there. If you wanted to render it, cool but no worries from me.

evamaxfield commented 2 years ago

This has been released. Install with napari-aicsimageio>=0.6.0

Thanks all :heart:

pr4deepr commented 2 years ago

Thanks for being really quick with this!