[BUG FIX] Zeiss .czi files not loading, pyvips error

Description

I have whole slide images (WSIs) scanned with a Zeiss scanner in file format .czi. I converted them to .ome.tiff with Bioformats (bfconvert) to be compatible with the pyvips library that Slideflow relies on. However, when I loaded in the WSI with sf.WSI() (with libvips backend) and tried WSI.otsu() or WSI.preview() I ran into an error with pyvips. I then used tiffinfo to examine the converted TIFF structure and realized that some sublayers of the .ome.tiff file did not have tile width present in their metadata. This revealed a fault during conversion with bfconvert where the default tile width was too large for the smaller subfiles. I resolved the issue by specifying the tilex and tiley sizes during conversion.

To Reproduce

I used the following code in the command line to convert the .czi file to .tiff: ./bfconvert 2023_11_01__8855.czi 2023_11_01__8855.ome.tiff

I then tried to load in the new .ome.tiff Slideflow code which caused an error:

import os
import slideflow as sf
os.environ['SF_SLIDE_BACKEND'] = 'libvips'
WSI_img = sf.WSI(path=slidepath,
       tile_px=299,
       tile_um=302,
       mpp=0.5,
       )
WSI_img.preview()

The error message:

Traceback (most recent call last):
  File "/home/pearsonlab/anaconda3/envs/sf/lib/python3.8/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/pearsonlab/anaconda3/envs/sf/lib/python3.8/multiprocessing/pool.py", line 48, in mapstar
    return list(map(*args))
  File "/home/pearsonlab/anaconda3/envs/sf/lib/python3.8/site-packages/slideflow/slide/backends/__init__.py", line 12, in tile_worker
    return tile_worker(*args, **kwargs)
  File "/home/pearsonlab/anaconda3/envs/sf/lib/python3.8/site-packages/slideflow/slide/backends/vips.py", line 277, in tile_worker
    gs_fraction = hsv_region[1].relational_const(
  File "/home/pearsonlab/anaconda3/envs/sf/lib/python3.8/site-packages/pyvips/vimage.py", line 919, in call_function
    return pyvips.Operation.call(name, self, *args, **kwargs)
  File "/home/pearsonlab/anaconda3/envs/sf/lib/python3.8/site-packages/pyvips/voperation.py", line 282, in call
    raise Error('unable to call {0}'.format(operation_name))
pyvips.error.Error: unable to call avg
  tiff2vips: out of order read -- at line 711, but line 506 requested

Environment

sf.about() output:
Version: 2.2.0
Backend: torch
Slide Backend: libvips (8.12.2)

Resolution

I examined the converted .ome.tiff file with tiffinfo on the command line and compared it other TIFF files I had.

tiffinfo output from non-CZI converted tiff (from the first downsample layer):

TIFF Directory at offset 0x1443649386 (560c576a)
  Subfile Type: reduced-resolution image (1 = 0x1)
  Image Width: 77824 Image Length: 48128
  Tile Width: 512 Tile Length: 512 # Tile Width present here and for all other sublayers (not shown for conciseness)
  Bits/Sample: 8
  Compression Scheme: JPEG
  Photometric Interpretation: YCbCr
  YCbCr Subsampling: 2, 2
  Orientation: row 0 top, col 0 lhs
  Samples/Pixel: 3
  Planar Configuration: single image plane
  ImageDescription: level=1 mag=20 quality=80
  Software: Philips DP v1.0
  JPEG Tables: (574 bytes)

Compared to tiffinfo output from CZI converted .ome.tiff:

TIFF Directory at offset 0x2101352288 (7d401760)
  Image Width: 5140 Image Length: 6152
  Tile Width: 1024 Tile Length: 1024 # Tile Width available in lower resolution layer
  Resolution: 58132.8, 58132.8 pixels/cm
  Bits/Sample: 8
  Sample Format: unsigned integer
  Compression Scheme: None
  Photometric Interpretation: RGB color
  Samples/Pixel: 3
  Planar Configuration: single image plane
  ImageDescription: ImageJ=
hyperstack=true
images=3
channels=3
slices=1
frames=1
  Software: OME Bio-Formats 7.0.0
...
TIFF Directory at offset 0x2233473516 (852019ec)
  Image Width: 2570 Image Length: 3076 # no "Tile Width" line here
  Resolution: 58132.8, 58132.8 pixels/cm
  Bits/Sample: 8
  Sample Format: unsigned integer
  Compression Scheme: None
  Photometric Interpretation: RGB color
  Samples/Pixel: 3
  Rows/Strip: 1
  Planar Configuration: single image plane
  ImageDescription: ImageJ=
hyperstack=true
images=3
channels=3
slices=1
frames=1
  Software: OME Bio-Formats 7.0.0

The key difference being that for lower resolution, higher layer sub-images of the .ome.tiff pyramid, there was no "Tile Width" tag in the metadata. Given that the pyvips error occurred during a step with calling tile workers, I realized there was something wrong with subtiling of the OME-TIFF file during conversion with bfconvert.

After playing around with a few options, I realized that I could manually set the tilex and tiley during conversion (I also made sure to set only one timepoint to be used for all layers). I chose tilex=512 and tiley=512, which is smaller than the total x & y dimensions of all sublayers of the WSI. This allows all sublayers to be saved in a tiled format (see Bioformats docs for more info on this option). Ultimately, the following code worked to successfully convert the files:

./bfconvert -timepoint 0 -tilex 512 -tiley 512 2023_11_01__8855.czi 2023_11_01__8855_t0_tiles.ome.tiff

Loading with sf.WSI() and then WSI.preview() worked!

jamesdolezal / slideflow