Use `ome-types` to build OME data

[NHPatterson]

Description

Currently OME data is composed using tifffile. Using ome-types will provide a more complete OME vocabulary.

[KidElectric]

Might be having an issue related to this! In particular: openslide parses the tiff.ImageDescription metadata of WSI files but I don't think this is carried through in the output ome.tiff files with wsireg. To test this idea I used tifftools to manually copy tiff.ImageDescription metadata field into the registered ome.tiff output of wsireg:

wsi_raw = tifftools.read_tiff(svs_fn)
ome_reg = tifftools.read_tiff(reg_fn)
ome_reg['ifds'][0]['tags'][270] = wsi_raw['ifds'][0]['tags'][270]

In that case, openslide.OpenSlide() correctly parses the microscope / magnification metadata. Does ome-types solve this or would using tifftools / some other tiff manager as I tried above make sense in the wsiregpackage?

EDIT: (Or is it actually more complicated than this because the magnification / MPP data are effectively altered during registration and new values would need to be calculated?)

[NHPatterson]

This is a great discussion point... using ome-types would give a better API and more expansive OME model for wsireg which currently only uses a hack version of what tifffile offers. I wasn't aware that any pyramidal OME-TIFF was compatible with OpenSlide at all! It's relatively ancient software. I've avoided using it in wsireg because it is difficult to install and deploy and many of the formats it supports are also supported by tifffile. That said, I think if you copy over the original image description, you will lose the OME metadata which I believe is in the ImageDescription TIFF tag.

EDIT: (Or is it actually more complicated than this because the magnification / MPP data are effectively altered during registration and new values would need to be calculated?)

After registration, the wsireg default is to write images at the um/px spacing of the fixed image. However, all transforms are specified in microns rather than pixels and as such they can be ported to any resolution. With this in mind, you can override the resampled pixel spacing by setting output_res = desired_res in the preprocessing settings when modalities are added. This default is to meet the expected applications. I assumed users would want to visualize images together, and depending on their visualization software, it may not be aware of physical spacing so having a resampled image at the same resolution as the target image makes it so the image pair has the same size and has a pixel perfect pixel-wise overlay. This is also useful for deep learning applications which may stack multiple WSIs. That said, it's totally possible to do registration, find the correct registration and have a mix of spatial resolutions. Viewers like napari and vitessce support physical scaling of the data, although in napari it has to be set like so:

viewer = napari.Viewer()
viewer.add_image(wsi_01, scale=[0.5,0.5])
viewer.add_image(wsi_02, scale=[0.75,0.75])

Hope this helps.

[KidElectric]

Thank you so much! You are right -- I think copying the metadata as I described above corrupts the file for certain applications (QuPath did not like it) but oddly enough made the ome-tiff compatible with OpenSlide -based code. The use case is that I am registering a large stack of tissue and using HoverNet for cell segmentation/classification. Hovernet relies on openslide out of the box... so that's where that gets tied in. I guess the other strategy would be to use HoverNet first, convert the json to .geojson, then perform registration on WSI + qupath style .geojson with wsireg (awesome feature but I haven't tried it yet!).

Is there an example of how to direct wsireg to .geojson data for that use case? Thank you again for all of your insight and help!

[NHPatterson]

Try this from the docs: https://wsireg.readthedocs.io/en/latest/usage.html#transform-shapes-in-geojson-file-exported-from-qupath-using-wsireg2d

I would suggest making a mock .geojson from QuPath, then getting the HoverNet json and making sure you get the format conversion down. It's pretty easy to do in python. You can easily check by loading the .geojson into QuPath. If it loads, it'll work in wsireg too.

Here's a bit of code I use sometimes to go between python and QuPath geojson. Only works for 2D scenarios. Color is set the same for all annotations/detections but you can modify this function as needed.

import json
import numpy as np

def np_to_qp_geojson(shape_array : np.ndarray, annotation_name: str = "default", qp_type: str = "annotation") -> dict:
    """numpy array of polygon vertices to QuPath geojson
    """

    geojson_type = "Feature"
    geo_type = "Polygon"

    properties = {
        "classification": {"name": annotation_name, "colorRGB": -3140401},
        "isLocked": False,
        "object_type": qp_type
    }

    geojson_dict = {
        "type": geojson_type,
        "geometry": {
            "type": geo_type,
            "coordinates": [shape_array.tolist()],
        },
        "properties": properties,
    }
    return geojson_dict

# n.b. the polygon must be closed and first and last coordinate are the same
rect_array = np.array([[0,0],[100,0],[100,100],[0,100],[0,0]], dtype=np.int32)

# qp_type can be "annotation" or "detection" which have differnt functionality in QuPath
gj = np_to_qp_geojson(rect_array, annotation_name="box", qp_type="annotation")

# in this case only one annotation was created
# so it is thrown in a list for write, if you already have a list, remove the list bracketing below
with open("C:/temp/test-gj.geojson", "w") as f:
    json.dump([gj],f)

[KidElectric]

Thank you so much! Yes I was able to convert hovernet -> qupath compatible cell object .geojson so that is great. Thank you for pointing me towards the docs.

Out of curiosity, is it possible to register the geojson in wsireg using the output of reg_graph.save_transformations() (e.g. the [...]_transformations.json ?) something likeCreate reg_graph object -> add previously used modalities -> add_attachment_shapes, [add_reg_path using previously calculated transformations? ] -> reg_graph.transform_shapes(). is override_prepro capable of using this dict?

[NHPatterson]

Cool, glad that worked. I think you've gotten to the point where the main API is limited. It doesn't really support "round-trips" or reloading completed graphs. I do run into this issue in my own work so I often will just script with internal library functions to do what I need. I do have a branch with a configurable transforming API locally but I haven't had the time to get it fully implemented... mainly because I also decided it was time to make the API fully-based on pydantic and that made it a much larger problem.

For now I think these scripts should get the job done. Let me know if you have other Qs or need scripting examples for other functionality you find in the main API.

transforming shapes outside of main

from wsireg.reg_shapes import RegShapes
from wsireg.reg_transforms import RegTransformSeq

reg_shapes = RegShapes("path/to/shapes.geojson")
rts = RegTransformSeq("path/to/transformations.json")

# optionally set output spacing to the fixed image
# this may not be encoded in the transformations file if you
# changed some settings
rts.set_output_spacing((1.0,1.0))

reg_shapes.transform_shapes(rts)
reg_shapes.save_shape_data("path/to/output_shapes.geojson",transformed=True)

transforming images outside of main

from wsireg.reg_images.loader import reg_image_loader
from wsireg.reg_transforms import RegTransformSeq
from wsireg.writers.ome_tiff_writer import OmeTiffWriter

# 1.0 is the image pixel spacing, it assumes isotropic pixel spacings
reg_image = reg_image_loader("path/to/myimage_or_my_np_array.tiff", 1.0, channel_names=["ch1","ch2"])
rts = RegTransformSeq("path/to/transformations.json")

# optionally set output spacing to the fixed image
# this may not be encoded in the transformations file if you
# changed some settings
rts.set_output_spacing((1.0,1.0))

# setting rts = None essentially makes this an OME-TIFF converter
writer = OmeTiffWriter(reg_image, rts)

# this will write an image called "output_image_name_no_ext.ome.tiff" in the "output/image/dir" folder
writer.write_image_by_plane("output_image_name_no_ext", 
                            output_dir="output/image/dir",
                            write_pyramid=True,
                            tile_size=512)