For certain WSIs I am getting error - "FAST exception caught in Qt event handler Image level is too large to convert into a FAST image"

[lomshabhishek]

I have few WSIs, couple of them were added to FastPathology app successfully however for rest of the WSIs I am getting this error Size of all of them is similar and they are of same format ".otif"

FAST exception caught in Qt event handler Image level is too large to convert into a FAST image

[andreped]

I have never heard of the .otif extension. FAST and FastPathology only supports the formats OpenSlide supports, in addition to Olympus' cellSens VSI format (.vsi).

Which scanner produced these images? Are you maybe referring to OME-TIFF (in that case the extension should be ome.tif)? Perhaps you converted some WSIs to OME-TIFF, but wrote the wrong extension?

[lomshabhishek]

It's like tif only maybe proprietary to scanner device, some otif files are working fine. Used with DSA and other similar platforms as well including fastpath.

I think I found the issue, the files that I am using some of them have high image level What is the highest level that we support? How can I set it in fast?

 File "/home/abhishek/.local/lib/python3.11/site-packages/fast/fast.py", line 3440, in run
    return _fast.ProcessObject_run(self, executeToken)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Image level is too large to convert into a FAST image

On Thu, Dec 14, 2023, 4:36 PM André Pedersen @.***> wrote:

I have never heard of the .otif extension. FAST and FastPathology only supports the formats OpenSlide supports, in addition to Olympus' cellSens VSI format (.vsi).

Are you maybe referring to OME-TIFF (in that case the extension should be ome.tif)? Which scanner produced these images?

— Reply to this email directly, view it on GitHub https://github.com/AICAN-Research/FAST-Pathology/issues/91#issuecomment-1855643735, or unsubscribe https://github.com/notifications/unsubscribe-auth/BBHZA54ULDKF25XRGTQ4X73YJLMTZAVCNFSM6AAAAABAUHNYL6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJVGY2DGNZTGU . You are receiving this because you authored the thread.Message ID: @.***>

--

Confidentiality Notice: The information transmitted in this email is intended only for the person(s) or entity to which it is addressed and may contain proprietary, confidential and/or privileged material. If you have received this email in error, please contact the sender by replying and delete this email so that it is not recoverable. If you are not the intended recipient(s), any retention, review, disclosure, distribution, copying, printing, dissemination, or other use of, or the taking of any action in reliance upon, this information is strictly prohibited and without liability on our part. This email has been checked for any viral content, however we take no responsibility for the forwarding process of this email and recommend that this message and any attachment(s) are scanned by your system for any potential viral content.

[smistad]

This error happens if you try to extract an image pyramid level as a single image which is larger than 16k x 16k pixels.

Probably this error happens in your case because Fastpathology tries to create a thumbnail from the lowest resolution level in the image pyramid. Probably this image pyramid lowest level is larger than 16k x 16k.

Do you know the image pyramid sizes for the failing images? You can retrieve it manually with FAST/pyFAST

[lomshabhishek]

Using fast.ImagePyramid.getLevelForMagnification(magnification=20).connect(importer).run()? I think it needs correction

[smistad]

image = importer.runAndGetOutputData()
for i in range(image.getNrOfLevels()):
    print(image.getLevelWidth(i), image.getLevelHeight(i))

[lomshabhishek]

image = importer.runAndGetOutputData()
for i in range(image.getNrOfLevels()):
    print(image.getLevelWidth(i), image.getLevelHeight(i))

This is the response i got 91648 78336, definitely larger that 16k x 16k

Any suggestions if this can be fixed?

[smistad]

Ok, so you don't have an image pyramid. You have just a single level which is very large. You can fix it with FAST, but maybe better to fix it at the source of where you got the data? How did you get this data?

[lomshabhishek]

Ok, so you don't have an image pyramid. You have just a single level which is very large. You can fix it with FAST, but maybe better to fix it at the source of where you got the data? How did you get this data?

This is from a local hospital, I checked with them, they have upgraded the camera lens hence some of the files which were scanned with older lens are working, the ones with new lens are huge.

They are using optra scanners, they can connect me to the vendor point of contact to see how can this be solved at the source.

How can this be fixed using fast?

[andreped]

So, the final format returned is not a pyramidal image, which FAST expects no, @smistad? I find it strange that the scanner used would output this format by default.

The reason why software such as QuPath is able to read these large single-plane images, is that it converts the image to an image pyramid when reading the image the first time (if relevant). This can take very long for larger images, so it would be much better if these images were stored in the appropriate format from the beginning.

But I guess this is something that could be added to FAST? Perhaps for the images you are working with, this is fine, but for 200k x 160k images, I would think this would not be a good solution.

[smistad]

Hi @lomshabhishek

Here is a python script which can convert your tiled tiff to an tiled image pyramid tiff:

import fast

importer = fast.WholeSlideImageImporter.create('non-image-pyramid.tiff')

image = importer.runAndGetOutputData()

new_image = fast.ImagePyramid.create(image.getFullWidth(), image.getFullHeight(), 3, image.getLevelTileWidth(0), image.getLevelTileHeight(0))
new_image.setSpacing(image.getSpacing())

accessRead = image.getAccess(fast.ACCESS_READ)
accessWrite = new_image.getAccess(fast.ACCESS_READ_WRITE)

counter = 0
totalTiles = image.getLevelTilesX(0)*image.getLevelTilesY(0)
for tile_x in range(image.getLevelTilesX(0)):
    for tile_y in range(image.getLevelTilesY(0)):
        patch = accessRead.getPatchAsImage(0, tile_x, tile_y)

        accessWrite.setPatch(0, tile_x*image.getLevelTileWidth(0), tile_y*image.getLevelTileHeight(0), patch)
        print('Progress', round(100*counter/totalTiles))
        counter += 1

print('Done. Saving file..')
fast.TIFFImagePyramidExporter.create('pyramid.tiff').connect(new_image).run()
print('Done. Opening file')

# Test that it works
importer = fast.WholeSlideImageImporter.create('pyramid.tiff')

renderer = fast.ImagePyramidRenderer.create().connect(importer)

fast.SimpleWindow2D.create().connect(renderer).run()

[lomshabhishek]

What location would it save the pyramid.tiff file to?

Cant locate it in cwd or using find / -name pyramid.tiff

Also If I try to run another script from examples I get this error, might be because it is not able to locate the pyramid.tiff file, not sure.

generator = fast.PatchGenerator.create(
        256, 256,
        magnification=10,
        overlapPercent=0
    ).connect(importer) \
    .connect(1, tissueSegmentation)

Error
Segmentation fault (core dumped)

[smistad]

It should appear in the working directory. You can specify a full absolute path to the TIFFImagePyramidExporter, then you know exactly where it ends up. I tested this script with the CMU-1.svs file and it worked nicely.

[smistad]

Did the conversion script work for you @lomshabhishek ? Are you able to get the images into Fastpathology after converting them?

[lomshabhishek]

Tried with absolute path as well didn't work.

Then I tried this script which worked.

import pyvips

def convert_to_pyramid(input_path, output_path):
    # Load the WSI image
    image = pyvips.Image.new_from_file(input_path, access='sequential')

    # Write the image to a pyramid file format (like TIFF with pyramidal layers)
    image.tiffsave(output_path, pyramid=True, tile=True, compression='jpeg', bigtiff=True)

if __name__ == "__main__":
    input_wsi = 'WSI file path'  # Replace with your WSI file path
    output_pyramid = 'output path'  # Replace with desired output path
    convert_to_pyramid(input_wsi, output_pyramid)

[smistad]

Hm. Did the image appear on screen at end of the script, or did it never show up?

[lomshabhishek]

Running on a cloud server, so graphics are disabled.

[smistad]

Are you able to share one of the failing images?

[lomshabhishek]

I tried, seems very difficult to get the images.