ScalablePyramidBuilder creates blank image tile in top level

[rudikershaw]

The ScalablePyramidBuilder is producing transparent blank images at all levels of the pyramid for certain images. We are using the following code;

final ScalablePyramidBuilder pyramidBuilder =
    new ScalablePyramidBuilder(254, 1, modelImage.getImageFormat(), "dzi");
final File tiledImageDirectory = com.google.common.io.Files.createTempDir();
final FilesArchiver archiver = new DirectoryArchiver(tiledImageDirectory);
final PartialImageReader pir = new BufferedImageReader(image);
pyramidBuilder.buildPyramid(pir, "pyramid", archiver, 1);

I have attached an image that reproduces this bug below.

[rudikershaw]

Additionally, this bug is reproducible simply using java -jar pyramidio-cli-1.1.0.jar -i car-image.jpg -o test

[rudikershaw]

After some investigation it appears that the bug occurs when ImageResizingHelper#resizeImage determines that an image is using a colour model with a 32bpp density. It then uses a 32bpp specific method to resize the image (presumably as some kind of optimization).

It's not clear whether the bug is in the 32bpp specific resize method, or whether we are incorrectly determining that the colour model is 32bpp (when it isn't).

Either way, stripping out the colour model specific resize methods appears to fix the problem. We deleted these lines from ImageResizingHelper#resizeImage;

if (img.getType() == BufferedImage.TYPE_USHORT_GRAY) {
    return resize16bppImage(img, width, height);
}
if (img.getColorModel().getPixelSize() == 32) {
    return resize32bppImage(img, width, height);
}

I've not raised a pull request because I assume some of your users will need these (presumed) optimizations. I know there were some changes in Java 8 (which we are using) to the Image API. Specifically with the way colours are interpreted.

[avandecreme]

Thanks for your analysis. It has been a long time since I wrote that code but from what I remember the goal of the img.getColorModel().getPixelSize() == 32 test was to check if we were manipulating a 32bpp (TIFF) grayscale image. From what I remember img.getType() was returning BufferedImage.TYPE_USHORT_GRAY for 16bpp grayscale images but nothing precise (null?) for 32bpp grayscale images. Maybe other properties of the image could be tested before jumping in that branch.

As for why that branch was necessary, my guess would be that resizeImageGraphics2D doesn't work properly for 16bpp and 32bpp grayscale images (because the Java image API doesn't).

[rudikershaw]

Thanks for your response @avandecreme. We're currently running a patched version of 1.1.0 that simply has those two branches stripped out, so it relies solely on the Java image API that your current code falls back on.

If we can confirm that the image API is working as expected in Java 8+ for a variety of image types (including TIFF grayscale images as you suggest), would you be happy to accept a pull request to strip the alternatives out?

This would mean releasing future versions of your library which will require Java 8+ to run. This is all hypothetical of course, at least until we can confirm that it is working correctly.

[avandecreme]

Since I don't work at NIST anymore, I don't have merge rights on this repository anymore. Maybe @pbajcsy has an opinion on this.

Anyway, I doubt that the Java API supports 16bpp and 32bpp TIFF images. Most image viewers don't. Imagej is one which does (it is interestingly written in Java but doesn't use the Java image API). Here is a zip file containing one 16bpp and one 32bpp image in case you want to try. They are taken from this data set: https://isg.nist.gov/deepzoomweb/view/image/replica3

[pbajcsy]

I concur with @avandecreme that our initial test images were grayscale 8 BPP, 16 BPP and 32 BPP (uncompressed). According to TIFF specs for v. 6, the TIFF file format supports grayscale images with 1, 2, 4, 8, 12, 16, and 32 BPP but different libraries support subsets of the BPP options. During the time of pyramidio development, our readers/writers were based on ImageIO library in Java Platform SE 8 (ImageIO library: BMP,GIF,JPEG,PNG,WBMP) and additional plugins, such as Java Advanced Imaging Image jai_imageio-1.1.jar (JPEG2000,PNM,RAW, and TIFF).

To update the pyramidio code, we could either redesign the code for a higher version of Java Platform SE than 8 or redesign the code to use BioFormats - see https://www.openmicroscopy.org/bio-formats/. We have been using BioFormats library in many other projects because it supports 150 formats and it can store images as tiled TIFF for efficient processing. In addition, we accelerated the code in C/C++ using a parallel library and posted the code at https://github.com/usnistgov/WIPP-pyramid-plugin (although it might not have all options for running the code from a command line if the options are important to @rudikershaw).

[darwinjob]

https://github.com/darwinjob/pyramidio-bioformats We've been using the mix of pyramidio and bio-formats for years so I decided to publish this as a GitHub project. It works best for tiled TIFFs (because of the small memory footprint) but it should work for the other bio-formats supported images as well. This is my first GitHub project - any tips are appreciated :)