Incorrect Color Interpretation in OpenSlide When Changing JPEG Q Factor in pyvips

[JulienMassonnet]

Operating System:

• Ubuntu

Libvips Version:

• Libvips 8.15 with OpenSlide 4.0

Slide Format:

• DICOM, TIFF

Issue Details:

I am using pyvips to load a slide (A) and save it as a JPEG-compressed TIFF (B). When I use tiffsave with the default Q factor (75), I can correctly load slide (B) with OpenSlide. However, if I change the Q factor value, OpenSlide displays the YCbCr channels instead of RGB.

This issue occurs with slide (A) in both DICOM and TIFF formats.

For slides in TIFF format, the issue arises when loading with pyvips.Image.new_from_file or pyvips.Image.openslideload, but not with vips.Image.tiffload.

The problem in my case is that I want to load a DICOM and save it to TIFF.

Using tifftools to inspect slide (B)'s metadata, I found that the TIFF TAG PhotometricInterpretation (262) is set to 2 (RGB) for problematic slides and 6 (YCbCr) for slides that are read correctly. It appears that OpenSlide is not converting to RGB due to this metadata.

Manually changing this value allows OpenSlide to read the slide correctly, although the following warning is still generated:

OpenSlide Warning:

JPEGFixupTagsSubsamplingSec: Warning, Auto-corrected former TIFF subsampling values [2,2] to match subsampling values inside JPEG compressed data [1,1].

[choosehappy]

i stumbled across your issue and also found https://github.com/libvips/pyvips/issues/133 and it seems to say that one can use:

 x.get_fields()  # get list of fields
x.get('orientation') # find relevant field
x.set('orientation', 99) # modify

could this somehow be incorporated into your code to "override" the defaults, i.e., either using this to explicitly set the correct TAG or compression subsampling values?

wouldn't resolve the underlying issue - but might still enable working around for the time being?

[jcupitt]

Hello @JulienMassonnet,

libvips will automatically disable chroma subsample for Q >= 90 (imagemagick does this too).

For example:

$ vips copy k2.jpg x.tif[tile,pyramid,compression=jpeg,Q=89]
$ tiffinfo x.tif 
=== TIFF directory 0 ===
TIFF Directory at offset 0x45a28 (285224)
  Image Width: 1450 Image Length: 2048
  Tile Width: 128 Tile Length: 128
  Resolution: 72.009, 72.009 pixels/inch
  Bits/Sample: 8
  Sample Format: unsigned integer
  Compression Scheme: JPEG
  Photometric Interpretation: YCbCr
  Orientation: row 0 top, col 0 lhs
  Samples/Pixel: 3
  Planar Configuration: single image plane
  JPEG Tables: (574 bytes)
...

And with 90:

$ vips copy k2.jpg x.tif[tile,pyramid,compression=jpeg,Q=90]
$ tiffinfo x.tif 
=== TIFF directory 0 ===
TIFF Directory at offset 0x7fa7e (522878)
  Image Width: 1450 Image Length: 2048
  Tile Width: 128 Tile Length: 128
  Resolution: 72.009, 72.009 pixels/inch
  Bits/Sample: 8
  Sample Format: unsigned integer
  Compression Scheme: JPEG
  Photometric Interpretation: RGB color
  Orientation: row 0 top, col 0 lhs
  Samples/Pixel: 3
  Planar Configuration: single image plane
  JPEG Tables: (574 bytes)
...

You can see the photometric interpretation has changed. Even though the pixels are now RGB (not YCbCr), openslide will still run a YCbCr->RGB conversion, resulting in crazy colours.

The simplest fix is to stay under Q 90. Would this work for you?

[JulienMassonnet]

Thanks for your responses.

To give a little more context, our research pipeline needs to convert DICOM Whole Slide Images to pyramidal TIFF. We are currently evaluating various JPEG compression levels to determine the optimal balance for our specific use case.

Given that the source DICOM files are already compressed, we are aiming to minimize any additional compression artifacts introduced during the conversion process. We measured PSNR and entropy of the compression loss across different compression settings. We've observed that chroma subsampling significantly impacts the image signals, leading us to explore higher Q factors.

We have also noticed that a Q factor of 90 performs slightly better than Q factors in the range of 91-95, is there a reason for that? At Q=90, the PSNR of the WSI is 54.38 with the worst 1k x 1k tile reaching 45.72 dB. At Q=91, the PSNR of the WSI is 53.29 with the worst 1k x 1k tile reaching 45.27 dB while increasing the disk space by ~2%. I’ve also compared a region and saw that for Q=91, the entropy of the compression loss is ~0.2 higher for each channel.

You can see the photometric interpretation has changed. Even though the pixels are now RGB (not YCbCr), openslide will still run a YCbCr->RGB conversion, resulting in crazy colors.

Are you saying that openslide applies the YCbCr->RGB conversion to the RGB channel? Does it apply twice the conversion? When I change the photometric interpretation tag to YCbCr, openslide can correctly read the region, doesn't this imply that the conversion is needed? Shouldn’t this tag always be YCbCr here?

Regarding the openslide warning, I want to fix this metadata for stability purposes. We noticed that the TIFF files generated by pyvips lack the YCbCrSubSampling (530) tag. While the default value of [2,2] works with a Q factor below 90, when Q is above 90, hence with no chroma subsampling, it necessitates a tag value of [1,1]. Using image.set_type doesn't seem to be able to add extension TIFF tags.

[jcupitt]

You can copy the JPEG DICOM tiles out directly into libtiff, with a bit of work. Have you considered this? You could save all decompress/recompress, and it should be very quick. libdicom (the lib openslide uses for DICOM read) will let you get the raw JFIF tiles from the DICOM:

https://github.com/ImagingDataCommons/libdicom

You do need to be careful with photometric interpretation (ie. YCbCr vs, RGB). DICOM has JFIF tiles (not JPEG!) and these don't include the PI as part of the tile. You'll need to get the PI from the DICOM, then use that to set the PI in the tiff you write.

To add even more confusion, libtiff has a pseudotag (not a real tag) for the JPEG colour mode which you'll need to be aware of. If you're just copying tiles in, it's probably OK to avoid it.

Using image.set_type doesn't seem to be able to add extension TIFF tags.

No, libtiff does not support generic tag write.