TiffWritter converts multiple TiffData elements into a single TiffData element with PlaneCount=N channels.

[dougwood]

First, thank you for your very useful and powerful module. We particularly appreciate the feature of TiffWriter.write that allows us to pass a function to generate tiles instead of having to supply the entire image. But we are having a problem with TiffWriter's output in a particular application (Concentriq) that reads OME tiff data.

I am using tiffile.TiffWriter(path, ome=True, bigtiff=True) to produce an OME TIFF file and it while this is working successfully to produce a multichannel, pyramid tiff that QuPath and other applications can read, unfortunately the application we are trying to use (Concentriq) does not interpret the resulting OME metadata successfully.

In the call to TiffWriter.write() I am using the following for the metadata dictionary:

{'Creator': 'uvim 0.28.0', 'Instrument': 'instrument', 'Image': {'ID': 'Image:0', 'Name': 'FAS Tonsil#4 cropped_Rd 1.ome.tif', 'ObjectiveSettings': {'ID': 'Objective:1'}, 'Pixels': {'BigEndian': 'true', 'ID': 'Pixels:0', 'Type': 'uint16', 'PhysicalSizeX': '0.000325', 'PhysicalSizeXUnit': 'mm', 'PhysicalSizeY': '0.000325', 'PhysicalSizeYUnit': 'mm', 'Channel': [{'EmissionWavelength': '440', 'Color': '65535', 'ID': 'Channel:0:0', 'Name': 'DAPI', 'SamplesPerPixel': '1'}, {'EmissionWavelength': '382', 'Color': '-6749953', 'ID': 'Channel:0:1', 'Name': 'FITC', 'SamplesPerPixel': '1'}, {'EmissionWavelength': '415', 'Color': '-872480513', 'ID': 'Channel:0:2', 'Name': 'TRITC', 'SamplesPerPixel': '1'}, {'EmissionWavelength': '448', 'Color': '872349951', 'ID': 'Channel:0:3', 'Name': 'Cy5', 'SamplesPerPixel': '1'}, {'EmissionWavelength': '481', 'Color': '16738047', 'ID': 'Channel:0:4', 'Name': 'Cy7', 'SamplesPerPixel': '1'}, {'EmissionWavelength': '440', 'Color': '65535', 'ID': 'Channel:0:5', 'Name': 'DAPI2', 'SamplesPerPixel': '1'}, {'EmissionWavelength': '546', 'Color': '6750207', 'ID': 'Channel:0:6', 'Name': 'FITC2', 'SamplesPerPixel': '1'}, {'EmissionWavelength': '579', 'Color': '838926335', 'ID': 'Channel:0:7', 'Name': 'TRITC2', 'SamplesPerPixel': '1'}, {'EmissionWavelength': '612', 'Color': '-872349697', 'ID': 'Channel:0:8', 'Name': 'Cy52', 'SamplesPerPixel': '1'}, {'EmissionWavelength': '645', 'Color': '-16738049', 'ID': 'Channel:0:9', 'Name': 'Cy72', 'SamplesPerPixel': '1'}], 'TiffData': [{'FirstC': '0', 'FirstT': '0', 'FirstZ': '0', 'IFD': '0'}, {'FirstC': '1', 'FirstT': '0', 'FirstZ': '0', 'IFD': '1'}, {'FirstC': '2', 'FirstT': '0', 'FirstZ': '0', 'IFD': '2'}, {'FirstC': '3', 'FirstT': '0', 'FirstZ': '0', 'IFD': '3'}, {'FirstC': '4', 'FirstT': '0', 'FirstZ': '0', 'IFD': '4'}, {'FirstC': '5', 'FirstT': '0', 'FirstZ': '0', 'IFD': '5'}, {'FirstC': '6', 'FirstT': '0', 'FirstZ': '0', 'IFD': '6'}, {'FirstC': '7', 'FirstT': '0', 'FirstZ': '0', 'IFD': '7'}, {'FirstC': '8', 'FirstT': '0', 'FirstZ': '0', 'IFD': '8'}, {'FirstC': '9', 'FirstT': '0', 'FirstZ': '0', 'IFD': '9'}]}}}

Notice that at the end of this dictionary, there is a 'TiffData' key which is a List with 10 elements.

But when we examine the image description of the resulting tiff file, we find (after pretty print formatting):

<?xml version="1.0" encoding="UTF-8"?>

Notice that the TiffData list of 10 elements has been turned into a single TiffData element with PlaneCount="10". Apparently this conforms to the OME tiff standard, because many OME tiff readers, such as QuPath are happy with this, but the reader we need to use is not. It seems that it would prefer to see the following instead of :

    <TiffData FirstC="0" FirstT="0" FirstZ="0" IFD="0"/>
    <TiffData FirstC="1" FirstT="0" FirstZ="0" IFD="1"/>
    <TiffData FirstC="2" FirstT="0" FirstZ="0" IFD="2"/>
    <TiffData FirstC="3" FirstT="0" FirstZ="0" IFD="3"/>
    <TiffData FirstC="4" FirstT="0" FirstZ="0" IFD="4"/>
    <TiffData FirstC="5" FirstT="0" FirstZ="0" IFD="5"/>
    <TiffData FirstC="6" FirstT="0" FirstZ="0" IFD="6"/>
    <TiffData FirstC="7" FirstT="0" FirstZ="0" IFD="7"/>
    <TiffData FirstC="8" FirstT="0" FirstZ="0" IFD="8"/>
    <TiffData FirstC="9" FirstT="0" FirstZ="0" IFD="9"/>

I think this may conform to the OME standard as well as it is also readable by QuPath and others.

Can you please advise as to how I can prevent TiffWritter from condensing the list of TiffData elements into one?

Thank you, Doug

[cgohlke]

For reference, the TiffData element comes from tifffile.py#L15492 and complies with the OME specification.

The issue is a bug in the Concentriq software, which should be fixed. You could try to work around the issue:

• write an empty TiffData element, e.g. <TiffData />, which is the specification default and should work for single series files. Maybe Concentriq can read that element? If that works, it could be integrated into tifffile.
• provide your own OME-XML metadata, e.g., imwrite(..., ome=False, description=ome_xml_string.encode(), metadata=None).

[dougwood]

Thank you Christophe for your prompt response. I agree that it would be best to get Concentriq to fix this issue and we are pursuing that, but it may take some time to get a fix released and I would like to get on with a temporary workaround if possible. I tried option #1 but that was not successful. As for #2, I am having some difficulty getting that to work.

My code that generates a multichannel, tiled pyramid tiff that can be read by QuPath and others is the following:

    with tifffile.TiffWriter(self.path, ome=True, bigtiff=True) as tiff:
        tiff.write(
            data=self.generate_tiles(),
            metadata=metadata_dict,
            software=software.encode("utf-8"),
            shape=self.level_full_shapes[0],
            subifds=int(self.num_levels - 1),
            dtype=dtype,
            tile=self.tile_shapes[0],
            resolution=(resolution_cm, resolution_cm, "centimeter"),
            photometric="minisblack",
            compression="lzw",
        )
        if self.verbose:
            print("Generating pyramid")
        for level, (shape, tile_shape) in enumerate(
            zip(self.level_full_shapes[1:], self.tile_shapes[1:]), 1
        ):
            if self.verbose:
                print(f"    Level {level} ({shape[2]} x {shape[1]})")
            tiff.write(
                data=self.subres_tiles(level),
                shape=shape,
                subfiletype=1,
                dtype=dtype,
                tile=tile_shape,
                compression="lzw",
            )

It produces an ome.tif with the following image-description:

<?xml version="1.0" encoding="UTF-8"?>

To test option #2 without having to concern myself if I am generating the OME metadata correctly, I first tried copying the string above into a str variable named OME_xml and set keywords metadata to None and ome to False:

 with tifffile.TiffWriter(self.path, ome=False, bigtiff=True) as tiff:
        tiff.write(
            data=self.generate_tiles(),
            metadata=None,
            description=OME_xml.encode(),
            software=software.encode("utf-8"),
            shape=self.level_full_shapes[0],
            subifds=int(self.num_levels - 1),
            dtype=dtype,
            tile=self.tile_shapes[0],
            resolution=(resolution_cm, resolution_cm, "centimeter"),
            photometric="minisblack",
            compression="lzw",
        )
        if self.verbose:
            print("Generating pyramid")
        for level, (shape, tile_shape) in enumerate(
            zip(self.level_full_shapes[1:], self.tile_shapes[1:]), 1
        ):
            if self.verbose:
                print(f"    Level {level} ({shape[2]} x {shape[1]})")
            tiff.write(
                data=self.subres_tiles(level),
                shape=shape,
                subfiletype=1,
                dtype=dtype,
                tile=tile_shape,
                compression="lzw",
            )

But with these changes, the code successfully completes the first call to TiffWriter.write(), but the subsequent call generate the first level of the pyramid, I get the following exception:

Generating pyramid Level 1 (9216 x 8192) Traceback (most recent call last): File "C:\Users\Doug\AppData\Local\ ... \lib\site-packages\tifffile\tifffile.py", line 7412, in init raise ValueError(f'suspicious number of tags {tagno}') ValueError: suspicious number of tags 34362575177

Your suggestion mentioned calling imwrite() not TiffWriter.write() so perhaps this is the reason for the exception.

Again, thank you very much for your help with this.

[cgohlke]

Strange, ValueError: suspicious number of tags is an error raised by the TIFF reader. Can you post the full traceback?

[cgohlke]

I can't reproduce the error:

import numpy
import tifffile

dtype = 'uint16'
shapes = (200, 200), (100, 100), (50, 50)
tile = (32, 32)
metadata_dict = {}
photometric = 'minisblack'

def generate_tiles():
    while True:
        yield numpy.ones(tile, dtype)

with tifffile.TiffWriter('issue189.ome.tif', ome=True, bigtiff=True) as tif:
    tif.write(
        data=generate_tiles(),
        metadata=metadata_dict,
        shape=shapes[0],
        subifds=len(shapes) - 1,
        dtype=dtype,
        tile=tile,
        photometric=photometric,
        compression='lzw',
    )
    for level in range(len(shapes) - 1):
        tif.write(
            data=generate_tiles(),
            shape=shapes[level + 1],
            subfiletype=1,
            dtype=dtype,
            tile=tile,
            photometric=photometric,
            compression='lzw',
        )

with tifffile.TiffFile('issue189.ome.tif') as tif:
    omexml = tif.pages.first.description

with tifffile.TiffWriter('issue189.tif', ome=False, bigtiff=True) as tif:
    tif.write(
        data=generate_tiles(),
        description=omexml.encode(),
        metadata=None,
        shape=shapes[0],
        subifds=len(shapes) - 1,
        dtype=dtype,
        tile=tile,
        photometric=photometric,
        compression='lzw',
    )
    for level in range(len(shapes) - 1):
        tif.write(
            data=generate_tiles(),
            shape=shapes[level + 1],
            subfiletype=1,
            dtype=dtype,
            tile=tile,
            photometric=photometric,
            compression='lzw',
        )

[cgohlke]

I tried option 1 but that was not successful.

Try other variations, e.g., <TiffData FirstC="0" FirstT="0" FirstZ="0" IFD="0" /> or <TiffData FirstC="0" FirstT="0" FirstZ="0" IFD="0" PlaneCount="10" />