search

ome / bioformats

Bio-Formats is a Java library for reading and writing data in life sciences image file formats. It is developed by the Open Microscopy Environment. Bio-Formats is released under the GNU General Public License (GPL); commercial licenses are available from Glencoe Software.
https://www.openmicroscopy.org/bio-formats
GNU General Public License v2.0
376 stars 242 forks source link

Bioformats unable to read image pyramid generated by VIPS 8.7.0 #3300

Open ebremer opened 5 years ago

ebremer commented 5 years ago

I've created a tiled Bigtiff image with an image pyramid using VIPS 8.7.0. I tried reading the image pyramid via an IIIF server I have written based on Bioformats. Bioformats is unable to see the image pyramid. I recreated the Bigtiff and pyramid using Bioformats 6.0.0-m3 and my IIIF server (which uses Bioformats to read images) is able to display the image using OpenSeadragon which I use in combination with my IIIF server. If I look at the VIPS generated tiled pyramid Bigtiff with GIMP, GIMP is able to show a series of images that shows the successfully smaller images of the image pyramid. Should Bioformats be able to see the VIPS pyramid or does VIPS have a problem?

dgault commented 5 years ago

Hi @ebremer, firstly apologies for the delayed response as the team has been on holiday. Do you have a link to the code you are using for the reading? Bio-Formats should I be able to read the various sub resolutions, depending on the setting for flattened resolutions these should either appear as separate images similar to GIMP (when using reader.setFlattenedResolutions(true);) or can be accessed using setResolution as in https://github.com/openmicroscopy/bioformats/blob/develop/components/formats-gpl/utils/SubResolutionExample.java (when using reader.setFlattenedResolutions(false);)

If there is still an issue accessing the sub resolutions then it may be best to send us a sample file to test.

ebremer commented 5 years ago

Hi @dgault, no worries! I hope you and your family had a great holiday. I tried out the above commands and didn't seem to have any success. You can re-create my problem with the following: 1) download the same SVS file: wget http://openslide.cs.cmu.edu/download/openslide-testdata/Aperio/CMU-2.svs 2) compile vips 8.7.3 (I updated it) with support for openslide. I installed openslide 3.4.1 with the developer headers so that when VIPS built it was recognized. 3) convert the above svs with the following: vips openslideload CMU-2.svs vipsCMU-2.tif[tile,compression=lzw,bigtiff,tile_width=256,tile_height=256,pyramid] 4) You can see the pyramid with: openslide-show-properties vipsCMU-2.tif 5) However, when you use bftools v5.9.2, it does not show the multiple resolutions. I used the code from your above SubResolutionExample.java (using Bioformats 6.0.0-m4) to try to see the multiple resolutions, but to no avail. 6) If I convert the image from svs to tif using bfconvert just to get it into tiff format, I can use ImageMagick's convert command to generate an image pyramid that IS recognized by Bioformats: convert CMU-2.tif -define tiff:tile-geometry=256x256 -compress lzw 'ptif:imCMU-2.tif' This tiff show all image pyramid levels using Bioformats. It appears that only the vips generated image pyramid isn't visible by Bioformats. Any help would be appreciated as I am trying to finish an iiif-based (https://iiif.io/) Image server using Bioformats has the core image library. The IIP image server works (http://iipimage.sourceforge.net/), but I want an all-java image server that does not require modification to handle whole slide images. The IIP image server I used has OpenSlide hacked into it to handle WSI.

ebremer commented 5 years ago

Here is a link to the file (CMU-2.svs ) already converted using the above process. Gimp and OpenSlide can see the other resolutions, but I cannot get Bioformats to see anything but the highest resolution.

https://drive.google.com/file/d/11HA6fUXVd42-DqIpOXNj7InP3k0qzwmL/view?usp=sharing

melissalinkert commented 5 years ago

Thank you for uploading a test file, @ebremer. I can reproduce the problem; the main issue is that the smaller pyramid resolutions have a SubfileType of 1 (see https://www.awaresystems.be/imaging/tiff/tifftags/newsubfiletype.html), which Bio-Formats handles differently.

A full write-up of what we need to fix is in this card on our issue tracking system, but unfortunately there isn't a workaround at the moment: https://trello.com/c/pW0zbhAK/306-detect-tiff-pyramids-generated-by-vips

ebremer commented 5 years ago

@melissalinkert please let me know if I can help do any testing. The life I save may be my own. ;)

jcupitt commented 5 years ago

Hello, I'm one of the libvips maintainers, I'd like to help improve compatibility if I can.

libvips (and imagemagick) just store pyramid levels as consecutive pages. libvips did write nested ifds with the subresolution tag at one point back in the mid 90s, but no one else could read them. Imagemagick went for simple consecutive pages, so we switched to that.

manics commented 5 years ago

@jcupitt We've got an intro guide on how to contribute to BioFormats: https://docs.openmicroscopy.org/bio-formats/6.0.1/developers/index.html#contributing-to-bio-formats

If you're interested in a more general discussion https://forum.image.sc/tags/bio-formats is probably the best place.

jcupitt commented 5 years ago

Could someone point me at docs, or perhaps a sample of the bioformats pyrtiff structure?

Here's how libvips does it, using a simple 1450x2048 jpg image as a source:

$ vips copy k2.jpg x.tif[pyramid,tile]
$ tiffinfo x.tif 
TIFF Directory at offset 0x900008 (9437192)
  Image Width: 1450 Image Length: 2048
  Tile Width: 128 Tile Length: 128
  Resolution: 28.346, 28.346 pixels/cm
  Bits/Sample: 8
  Sample Format: unsigned integer
  Compression Scheme: None
  Photometric Interpretation: RGB color
  Orientation: row 0 top, col 0 lhs
  Samples/Pixel: 3
  Planar Configuration: single image plane
TIFF Directory at offset 0xb406ea (11798250)
  Subfile Type: reduced-resolution image (1 = 0x1)
  Image Width: 725 Image Length: 1024
  Tile Width: 128 Tile Length: 128
  Resolution: 28.346, 28.346 pixels/cm
  Bits/Sample: 8
  Compression Scheme: None
  Photometric Interpretation: RGB color
  Orientation: row 0 top, col 0 lhs
  Samples/Pixel: 3
  Planar Configuration: single image plane
... etc.
TIFF Directory at offset 0xc0cb7e (12635006)
  Subfile Type: reduced-resolution image (1 = 0x1)
  Image Width: 90 Image Length: 128
  Tile Width: 128 Tile Length: 128
  Resolution: 28.346, 28.346 pixels/cm
  Bits/Sample: 8
  Compression Scheme: None
  Photometric Interpretation: RGB color
  Orientation: row 0 top, col 0 lhs
  Samples/Pixel: 3
  Planar Configuration: single image plane

So it keeps shrinking until the image fits within a tile. Layers are in consecutive pages, with the "reduced resolution" bit set.

ImageMagick does almost the same:

$ convert k2.jpg -define tiff:tile-geometry=128x128 ptif:x.tif
$ tiffinfo x.tif 
TIFF Directory at offset 0x900008 (9437192)
  Subfile Type: multi-page document (2 = 0x2)
  Image Width: 1450 Image Length: 2048
  Tile Width: 128 Tile Length: 128
  Resolution: 28.346, 28.346 pixels/cm
  Bits/Sample: 8
  Compression Scheme: None
  Photometric Interpretation: RGB color
  FillOrder: msb-to-lsb
  Orientation: row 0 top, col 0 lhs
  Samples/Pixel: 3
  Planar Configuration: single image plane
  Page Number: 0-6
  White Point: 0.3127-0.329
  PrimaryChromaticities: 0.640000,0.330000,0.300000,0.600000,0.150000,0.060000
TIFF Directory at offset 0xb40754 (11798356)
  Subfile Type: multi-page document (2 = 0x2)
  Image Width: 725 Image Length: 1024
  Tile Width: 128 Tile Length: 128
  Resolution: 14.173, 14.173 pixels/cm
  Bits/Sample: 8
  Compression Scheme: None
  Photometric Interpretation: RGB color
  FillOrder: msb-to-lsb
  Orientation: row 0 top, col 0 lhs
  Samples/Pixel: 3
  Planar Configuration: single image plane
  Page Number: 0-6
  White Point: 0.3127-0.329
  PrimaryChromaticities: 0.640000,0.330000,0.300000,0.600000,0.150000,0.060000
... etc.
TIFF Directory at offset 0xc18e84 (12684932)
  Subfile Type: multi-page document (2 = 0x2)
  Image Width: 45 Image Length: 64
  Tile Width: 128 Tile Length: 128
  Resolution: 0.885813, 0.885813 pixels/cm
  Bits/Sample: 8
  Compression Scheme: None
  Photometric Interpretation: RGB color
  FillOrder: msb-to-lsb
  Orientation: row 0 top, col 0 lhs
  Samples/Pixel: 3
  Planar Configuration: single image plane
  Page Number: 0-6
  White Point: 0.3127-0.329
  PrimaryChromaticities: 0.640000,0.330000,0.300000,0.600000,0.150000,0.060000

Except it writes one layer more (I guess IM is using < rather <= to detect when the image fits within a tile), and IM doesn't set the reduced resolution bit. Layers are also in simple consecutive pages. One curiosity is that IM changes the shrink factor slightly between layers to avoid fractional pixels at the bottom and right (libvips always does a strict /2, with round up).

libvips and imagemagick can read each other's tiff pyramids, and both work with iipimage.

sbesson commented 5 years ago

Hi @jcupitt, one of the best representative example of a tiled pyramidal TIFF file using multi-layers supported by Bio-Formats is a large stitched TEM image published in JCB. This image was published in the Image Data Resource and the raw data can be downloaded using Aspera although note the file is 500GB.

For the sake of discussion, the TIFF is constructed as follows:

[sbesson@idr0-slot3 ~]$ tiffinfo /uod/idr/filesets/idr0053-faas-virtualnanoscopy/20181128-ftp/figures_jcb_submission_faas_et_al/fig1/fig1.tif 
TIFF Directory at offset 0x51bd000010 (351063244816)
  Image Width: 921600 Image Length: 380928 Image Depth: 1
  Tile Width: 256 Tile Length: 256
  Resolution: 9.31323e-10, 9.31323e-10
  Bits/Sample: 8
  Compression Scheme: None
  Photometric Interpretation: min-is-black
  Orientation: row 0 bottom, col 0 lhs
  Samples/Pixel: 1
  Planar Configuration: single image plane
  Software: 
   Section Electron Microscopy
   Department Molecular Cell Biology
   Leiden University Medical Center
   Frank Faas, 2010
  Artist: De kaboutertjes
TIFF Directory at offset 0x66315bd202 (438914765314)
  Image Width: 460800 Image Length: 190464 Image Depth: 1
  Tile Width: 256 Tile Length: 256
  Resolution: 9.31323e-10, 9.31323e-10
  Bits/Sample: 8
  Compression Scheme: None
  Photometric Interpretation: min-is-black
  Orientation: row 0 bottom, col 0 lhs
  Samples/Pixel: 1
  Planar Configuration: single image plane
  Software: 
   Section Electron Microscopy
   Department Molecular Cell Biology
   Leiden University Medical Center
   Frank Faas, 2010
  Artist: De kaboutertjes
TIFF Directory at offset 0x6b4e72c7f4 (460877645812)
  Image Width: 230400 Image Length: 95232 Image Depth: 1
  Tile Width: 256 Tile Length: 256
  Resolution: 9.31323e-10, 9.31323e-10
  Bits/Sample: 8
  Compression Scheme: None
  Photometric Interpretation: min-is-black
  Orientation: row 0 bottom, col 0 lhs
  Samples/Pixel: 1
  Planar Configuration: single image plane
  Software: 
   Section Electron Microscopy
   Department Molecular Cell Biology
   Leiden University Medical Center
   Frank Faas, 2010
  Artist: De kaboutertjes
TIFF Directory at offset 0x6c95b886e6 (466368366310)
  Image Width: 115200 Image Length: 47616 Image Depth: 1
  Tile Width: 256 Tile Length: 256
  Resolution: 9.31323e-10, 9.31323e-10
  Bits/Sample: 8
  Compression Scheme: None
  Photometric Interpretation: min-is-black
  Orientation: row 0 bottom, col 0 lhs
  Samples/Pixel: 1
  Planar Configuration: single image plane
  Software: 
   Section Electron Microscopy
   Department Molecular Cell Biology
   Leiden University Medical Center
   Frank Faas, 2010
  Artist: De kaboutertjes
TIFF Directory at offset 0x6ce789f818 (467741046808)
  Image Width: 57600 Image Length: 23808 Image Depth: 1
  Tile Width: 256 Tile Length: 256
  Resolution: 9.31323e-10, 9.31323e-10
  Bits/Sample: 8
  Compression Scheme: None
  Photometric Interpretation: min-is-black
  Orientation: row 0 bottom, col 0 lhs
  Samples/Pixel: 1
  Planar Configuration: single image plane
  Software: 
   Section Electron Microscopy
   Department Molecular Cell Biology
   Leiden University Medical Center
   Frank Faas, 2010
  Artist: De kaboutertjes
TIFF Directory at offset 0x6cfc4e15da (468089443802)
  Image Width: 28800 Image Length: 11904 Image Depth: 1
  Tile Width: 256 Tile Length: 256
  Resolution: 9.31323e-10, 9.31323e-10
  Bits/Sample: 8
  Compression Scheme: None
  Photometric Interpretation: min-is-black
  Orientation: row 0 bottom, col 0 lhs
  Samples/Pixel: 1
  Planar Configuration: single image plane
  Software: 
   Section Electron Microscopy
   Department Molecular Cell Biology
   Leiden University Medical Center
   Frank Faas, 2010
  Artist: De kaboutertjes
TIFF Directory at offset 0x6d01a763bc (468179182524)
  Image Width: 14400 Image Length: 5952 Image Depth: 1
  Tile Width: 256 Tile Length: 256
  Resolution: 9.31323e-10, 9.31323e-10
  Bits/Sample: 8
  Compression Scheme: None
  Photometric Interpretation: min-is-black
  Orientation: row 0 bottom, col 0 lhs
  Samples/Pixel: 1
  Planar Configuration: single image plane
  Software: 
   Section Electron Microscopy
   Department Molecular Cell Biology
   Leiden University Medical Center
   Frank Faas, 2010
  Artist: De kaboutertjes
TIFF Directory at offset 0x6d0303bb2e (468202011438)
  Image Width: 7200 Image Length: 2976 Image Depth: 1
  Tile Width: 256 Tile Length: 256
  Resolution: 9.31323e-10, 9.31323e-10
  Bits/Sample: 8
  Compression Scheme: None
  Photometric Interpretation: min-is-black
  Orientation: row 0 bottom, col 0 lhs
  Samples/Pixel: 1
  Planar Configuration: single image plane
  Software: 
   Section Electron Microscopy
   Department Molecular Cell Biology
   Leiden University Medical Center
   Frank Faas, 2010
  Artist: De kaboutertjes
TIFF Directory at offset 0x6d035dd2e0 (468207915744)
  Image Width: 3600 Image Length: 1488 Image Depth: 1
  Tile Width: 256 Tile Length: 256
  Resolution: 9.31323e-10, 9.31323e-10
  Bits/Sample: 8
  Compression Scheme: None
  Photometric Interpretation: min-is-black
  Orientation: row 0 bottom, col 0 lhs
  Samples/Pixel: 1
  Planar Configuration: single image plane
  Software: 
   Section Electron Microscopy
   Department Molecular Cell Biology
   Leiden University Medical Center
   Frank Faas, 2010
  Artist: De kaboutertjes
TIFF Directory at offset 0x6d0375da72 (468209490546)
  Image Width: 1800 Image Length: 744 Image Depth: 1
  Tile Width: 256 Tile Length: 256
  Resolution: 9.31323e-10, 9.31323e-10
  Bits/Sample: 8
  Compression Scheme: None
  Photometric Interpretation: min-is-black
  Orientation: row 0 bottom, col 0 lhs
  Samples/Pixel: 1
  Planar Configuration: single image plane
  Software: 
   Section Electron Microscopy
   Department Molecular Cell Biology
   Leiden University Medical Center
   Frank Faas, 2010
  Artist: De kaboutertjes
TIFF Directory at offset 0x6d037ddde4 (468210015716)
  Image Width: 900 Image Length: 372 Image Depth: 1
  Tile Width: 256 Tile Length: 256
  Resolution: 9.31323e-10, 9.31323e-10
  Bits/Sample: 8
  Compression Scheme: None
  Photometric Interpretation: min-is-black
  Orientation: row 0 bottom, col 0 lhs
  Samples/Pixel: 1
  Planar Configuration: single image plane
  Software: 
   Section Electron Microscopy
   Department Molecular Cell Biology
   Leiden University Medical Center
   Frank Faas, 2010
  Artist: De kaboutertjes

As noted in the initial investigation, the PyramidTiffReader is used to detect and read this particular file layout but most of its logic could be reused to read libvips/imagemagick-generated pyramidal TIFF files as well.

If you or someone else anted to try out extending this reader to support generic tiled multi-resolution TIFF files, let us know if we can give additional pointers. In addition, our review progress includes a CI infrastructure performing non-regression daily tests against sample files across all supported formats (many of them being TIFF variations).

jcupitt commented 5 years ago

OK, I've set it downloading.

I have a fast university connection, but that download tool has topped out at 10 mbps, so (I think?) it'll take five days. We use http://academictorrents.com/ for our large data releases, I don't suppose there's a copy on there?

A smaller sample image would be helpful too, heh.

sbesson commented 5 years ago

A smaller sample image would be helpful too, heh.

Definitely, the above was primarily meant to reflect the largest driver for this particular file format/reader. I am sure there are some smaller samples under the idr0053 submission e.g. try the file available under 20181128-ftp/figures_jcb_submission_faas_et_al/fig6/components/els_green.tif which is 6GB large.

imagesc-bot commented 3 years ago

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/writing-qupath-readable-large-images-with-bio-formats/50715/16

imagesc-bot commented 3 years ago

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/qupath-not-loading-pyramid-tiffs-properly/51221/8

jcupitt commented 3 years ago

libvips 8.10 now has support for subifd TIFF pyramids, so I think this issue can be closed. The forum posts imagesc-bot linked above have sample conversion scripts.

ebremer commented 3 years ago

Thanks @jcupitt ! Let me give it a whirl and verify.