jai-imageio / jai-imageio-core

JAI ImageIO Core (without javax.media.jai dependencies)
Other
234 stars 87 forks source link

Corrupt TIFFs created for incompressible data using zlib compression #68

Open twotabbies opened 3 years ago

twotabbies commented 3 years ago

https://github.com/jai-imageio/jai-imageio-core/blob/master/src/main/java/com/github/jaiimageio/impl/plugins/tiff/TIFFDeflater.java#L90 counts the number of 32Ki blocks based on the input size, and then allocates a compression buffer assuming that zlib compression will, in the worst case, add at most 5 bytes per 32k block, plus a 6 byte header. This is incorrect: https://www.zlib.net/zlib_tech.html says "the only expansion is an overhead of five bytes per 16 KB

block (about 0.03%), plus a one-time overhead of six bytes for the entire stream" - note the block size!

If the input data is incompressible, the resulting TIFF file is corrupt as the zlib data is truncated, due to the buffer being too small. An example file is attached (gzipped because github won't accept a plain .tif). test.tif.gz

schwehr commented 3 years ago

From @twotabbies fix:

int blocks = (inputSize + 32767)/32768;

// Worst case for Zlib deflate is input size + 5 bytes per 32k

Should be

int blocks = (inputSize + 16383)/16384;

// Worst case for Zlib deflate is input size + 5 bytes per 16k

Patch style:


--- a/imageio_ext/plugin/tiff/src/main/java/it/geosolutions/imageioimpl/plugins/tiff/TIFFDeflater.orig
+++ b/imageio_ext/plugin/tiff/src/main/java/it/geosolutions/imageioimpl/plugins/tiff/TIFFDeflater.java
@@ -116,10 +116,12 @@ public class TIFFDeflater extends TIFFCo
                       int scanlineStride) throws IOException {

         int inputSize = height*scanlineStride;
-        int blocks = (inputSize + 32767)/32768;
+        int blocks = (inputSize + 16383)/16384;

-        // Worst case for Zlib deflate is input size + 5 bytes per 32k
+        // Worst case for Zlib deflate is input size + 5 bytes per 16k
         // block, plus 6 header bytes
         byte[] compData = new byte[inputSize + 5*blocks + 6];

         int numCompressedBytes = 0;