[BUG] Compression scheme 33003 tile decoding is not implemented.

[OlivierDehaene]

Describe the bug

For Aperio svs files, I run into a cryptic error:

/slide.svs: Compression scheme 33003 tile decoding is not implemented.

However, cuCMI still loads a tile and doesn't raise but the color are wrong or it is entirely black. See the following grid of tiles:

Are all OpenSlide vendors supported? Is there a list of all supported vendors somewhere? I could not find it.

Steps/Code to reproduce bug

# Download the TCGA-F4-6459-01Z-00-DX1 slide from https://portal.gdc.cancer.gov/cases/9fd08502-355b-4f5a-a25c-73ec7184f6d3?bioId=57f4136d-d6aa-4537-ba73-f21fe0374005 

import numpy as np

from PIL import Image
from cucmi import CuImage

slide = CuImage("TCGA-F4-6459-01Z-00-DX1")

tile = slide.read_region((0, 0), (224, 224), 0)

Image.fromarray(np.asarray(tile))

[gigony]

Hello @OlivierDehaene ! Thank you for your interest in cuCIM!

Compression scheme 33003 is actually JPEG2000 which is used in Aperio SVS format.

cuCIM currently supports tiled RGB TIFF image(Generic TIFF or Philips TIFF) with JPEG or Deflate compressed and doesn't support SVS format yet. However, we have a plan to support it in the near future.

The tentative plan is specified in the following link (I would move the content to Github wiki page later): https://github.com/rapidsai/cucim/blob/branch-0.20/python/cucim/docs/roadmap/index.md#facalendar-alttext-info-mr-1-v020

If cuCIM encounters TIFF (SVS is a variant of TIFF format) file that is not JPEG/Deflate compressed RGB format, it just delegates its process to libtiff's TIFFRGBAImageGet() method and it seems that warning/error are not properly handled.

Let me fix the issue by raising an error until cuCIM supports SVS format properly. (I am currently working on providing a cache mechanism to cuCIM so that the performance could be greatly improved)

Thank you for the report! We are also eager to support various image formats so stay tuned!

[OlivierDehaene]

Thank you for your answer @gigony!

However, we have a plan to support it in the near future.

Amazing! I have been looking for a better alternative to OpenSlide for a while now and if cuCIM starts to support Aperio that would be great :)

I am currently working on providing a cache mechanism to cuCIM so that the performance could be greatly improved

Will we be able to disable the cache when doing random tile reads? We generally disable the OpenSlide one when training.

[gigony]

@OlivierDehaene

Will we be able to disable the cache when doing random tile reads? We generally disable the OpenSlide one when training.

I thought cuCIM can allocate some memory for cache by default and the user can increase the cache size programmatically. I assumed it wouldn't be a problem if we use a cache memory per 'parent process' and cache memory is shared among subprocesses (whereas OpenSlide seems to allocate cache memory per file).

However, you're right. It is better to have an option to disable cache at all as cache wouldn't much help when reading random partial images from a huge image pool during training.

Now I am thinking of allocating cache memory only when requested :)

Thank you so much for your valuable feedback!

[jakirkham]

Some packages should be pushed to rapidsai-nightly soon with Gigon's fix. Also this should show up in the next release. Please let us know if you run into any more issues 🙂