search

python-pillow / Pillow

Python Imaging Library (Fork)
https://python-pillow.org
Other
12.35k stars 2.24k forks source link

ValueError: Decompressed Data Too Large #8363

Closed robonrrd closed 2 months ago

robonrrd commented 2 months ago

What did you do?

I attempted to open a PNG, created with Pillow.

What did you expect to happen?

For the image to be successfully opened.

What actually happened?

An exception was thrown: ValueError: Decompressed Data Too Large

What are your OS, Python and Pillow versions?

--------------------------------------------------------------------
Pillow 9.5.0
Python 3.10.12 (main, Jul 29 2024, 16:56:48) [GCC 11.4.0]
--------------------------------------------------------------------
Python modules loaded from /home/jeffreys/.local/lib/python3.10/site-packages/PIL
Binary modules loaded from /home/jeffreys/.local/lib/python3.10/site-packages/PIL
--------------------------------------------------------------------
--- PIL CORE support ok, compiled for 9.5.0
*** TKINTER support not installed
--- FREETYPE2 support ok, loaded 2.13.0
--- LITTLECMS2 support ok, loaded 2.15
--- WEBP support ok, loaded 1.3.0
--- WEBP Transparency support ok
--- WEBPMUX support ok
--- WEBP Animation support ok
--- JPEG support ok, compiled for libjpeg-turbo 2.1.5.1
--- OPENJPEG (JPEG2000) support ok, loaded 2.5.0
--- ZLIB (PNG/ZIP) support ok, loaded 1.2.11
--- LIBTIFF support ok, loaded 4.5.0
--- RAQM (Bidirectional Text) support ok, loaded 0.10.0, fribidi 1.0.8, harfbuzz 7.1.0
*** LIBIMAGEQUANT (Quantization method) support not installed
--- XCB (X protocol) support ok
--------------------------------------------------------------------
BLP
Extensions: .blp
Features: open, save, encode
--------------------------------------------------------------------
BMP image/bmp
Extensions: .bmp
Features: open, save
--------------------------------------------------------------------
BUFR
Extensions: .bufr
Features: open, save
--------------------------------------------------------------------
CUR
Extensions: .cur
Features: open
--------------------------------------------------------------------
DCX
Extensions: .dcx
Features: open
--------------------------------------------------------------------
DDS
Extensions: .dds
Features: open, save
--------------------------------------------------------------------
DIB image/bmp
Extensions: .dib
Features: open, save
--------------------------------------------------------------------
EPS application/postscript
Extensions: .eps, .ps
Features: open, save
--------------------------------------------------------------------
FITS
Extensions: .fit, .fits
Features: open, save
--------------------------------------------------------------------
FLI
Extensions: .flc, .fli
Features: open
--------------------------------------------------------------------
FPX
Extensions: .fpx
Features: open
--------------------------------------------------------------------
FTEX
Extensions: .ftc, .ftu
Features: open
--------------------------------------------------------------------
GBR
Extensions: .gbr
Features: open
--------------------------------------------------------------------
GIF image/gif
Extensions: .gif
Features: open, save, save_all
--------------------------------------------------------------------
GRIB
Extensions: .grib
Features: open, save
--------------------------------------------------------------------
HDF5
Extensions: .h5, .hdf
Features: open, save
--------------------------------------------------------------------
ICNS image/icns
Extensions: .icns
Features: open, save
--------------------------------------------------------------------
ICO image/x-icon
Extensions: .ico
Features: open, save
--------------------------------------------------------------------
IM
Extensions: .im
Features: open, save
--------------------------------------------------------------------
IMT
Features: open
--------------------------------------------------------------------
IPTC
Extensions: .iim
Features: open
--------------------------------------------------------------------
JPEG image/jpeg
Extensions: .jfif, .jpe, .jpeg, .jpg
Features: open, save
--------------------------------------------------------------------
JPEG2000 image/jp2
Extensions: .j2c, .j2k, .jp2, .jpc, .jpf, .jpx
Features: open, save
--------------------------------------------------------------------
MCIDAS
Features: open
--------------------------------------------------------------------
MIC
Extensions: .mic
Features: open
--------------------------------------------------------------------
MPEG video/mpeg
Extensions: .mpeg, .mpg
Features: open
--------------------------------------------------------------------
MSP
Extensions: .msp
Features: open, save, decode
--------------------------------------------------------------------
PCD
Extensions: .pcd
Features: open
--------------------------------------------------------------------
PCX image/x-pcx
Extensions: .pcx
Features: open, save
--------------------------------------------------------------------
PIXAR
Extensions: .pxr
Features: open
--------------------------------------------------------------------
PNG image/png
Extensions: .apng, .png
Features: open, save, save_all
--------------------------------------------------------------------
PPM image/x-portable-anymap
Extensions: .pbm, .pgm, .pnm, .ppm
Features: open, save
--------------------------------------------------------------------
PSD image/vnd.adobe.photoshop
Extensions: .psd
Features: open
--------------------------------------------------------------------
QOI
Extensions: .qoi
Features: open
--------------------------------------------------------------------
SGI image/sgi
Extensions: .bw, .rgb, .rgba, .sgi
Features: open, save
--------------------------------------------------------------------
SPIDER
Features: open, save
--------------------------------------------------------------------
SUN
Extensions: .ras
Features: open
--------------------------------------------------------------------
TGA image/x-tga
Extensions: .icb, .tga, .vda, .vst
Features: open, save
--------------------------------------------------------------------
TIFF image/tiff
Extensions: .tif, .tiff
Features: open, save, save_all
--------------------------------------------------------------------
WEBP image/webp
Extensions: .webp
Features: open, save, save_all
--------------------------------------------------------------------
WMF
Extensions: .emf, .wmf
Features: open, save
--------------------------------------------------------------------
XBM image/xbm
Extensions: .xbm
Features: open, save
--------------------------------------------------------------------
XPM image/xpm
Extensions: .xpm
Features: open
--------------------------------------------------------------------
XVTHUMB
Features: open
--------------------------------------------------------------------
img = Image.open('001041.png')

001041

radarhere commented 2 months ago

Hi. You are hitting a limit when opening images.

https://pillow.readthedocs.io/en/stable/reference/plugins.html#PIL.PngImagePlugin.MAX_TEXT_CHUNK

PIL.PngImagePlugin.MAX_TEXT_CHUNK = 1048576 Maximum decompressed size for a iTXt or zTXt chunk. Eliminates decompression bombs where compressed chunks can expand 1000x. See Text in PNG File Format.

You can increase this in your script like so.

from PIL import Image, PngImagePlugin
PngImagePlugin.MAX_TEXT_CHUNK = 2 * 1024 * 1024
Image.open('001041.png')
robonrrd commented 2 months ago

That fixes my proximate issue (I can now open this image), thank you.

However, I would still claim this is a bug. It was a PNG generated by Pillow and the naive user (me!) would expect Pillow to also be able to open it. Further, OpenCV (and GIMP and qiv, etc.) can open it with no adjustment of internal constants.

radarhere commented 2 months ago

Are you able to provide a self-contained example to show how you generated it?

robonrrd commented 2 months ago

I used some code that uses blenderproc to render 3D objects on top of background images. I used the OpenImages V7 dataset as a source of background images.

radarhere commented 2 months ago

The problematic part of the image is the ICC profile, which is just over 1mb. We are raising an error to try and prevent decompression bombs, where compressed data becomes excessively large when expanded.

Without a self-contained example, it's hard to go into further detail, but it would appear that something external to Pillow is providing a large ICC profile. I think there is an expectation that Pillow saves exactly the data that it is given. I considered splitting it into multiple iCCP chunks, but http://www.libpng.org/pub/png/spec/1.2/PNG-Chunks.html#C.iCCP doesn't say anything about concatenating.

We could raise an error when saving a large ICC profile, but I don't think you would find that helpful.

If there was a self-contained example, we could try and figure out where the ICC profile comes from, but otherwise, I don't think there is anything for us to do here.

radarhere commented 2 months ago

@robonrrd do you have any further thoughts?

robonrrd commented 2 months ago

I still feel that this is a bug, or at least a bad preset that causes problems. My reasoning is that Pillow is the only image manipulation library I use which has a problem with this image and, further, the error message is not instructive and the fix is non-intuitive. I would never have been able to diagnose this without your help.

Perhaps the easiest solution is to kick the can down the road and increase the default maximum decompressed size? If you think changing the default max size is bad, perhaps change the error message to suggest that the user increase the size.

radarhere commented 2 months ago

I've created #8391. That will change the error message to "Decompressed data too large for PngImagePlugin.MAX_TEXT_CHUNK".

If a user sees that, and searches our docs for "PngImagePlugin.MAX_TEXT_CHUNK", they will find https://pillow.readthedocs.io/en/stable/handbook/image-file-formats.html#png-opening

Individual compressed chunks are limited to a decompressed size of PngImagePlugin.MAX_TEXT_CHUNK, by default 1MB, to prevent decompression bombs.