Saving TIFF with compression throws Segmentation Fault 11 in 5.4.1, but not 5.3.0

[jeremydmoore]

What did you do?

Opened a bitonal TIFF created with Adobe Photoshop CC (version 20.0.2) then saved it with Group4 compression using Pillow.

TIFF Image should be downloadable from Github repository here: https://github.com/photosbyjeremy/images/blob/master/test_bitonal_text.tif.zip

What did you expect to happen?

I expected the image to save with compression.

What actually happened?

In Pillow 5.4.1 there was a Segmentation Fault 11, but in Pillow 5.3.0 it works just fine.

What are your OS, Python and Pillow versions?

• OS: macOS 10.14.3
• Python: 3.6.8 (and I actually tried 3.7.1 first, but didn't screenshot those attempts)
• Pillow: 5.4.1 and 5.3.0

Please include code that reproduces the issue and whenever possible, an image that demonstrates the issue. Please upload images to GitHub, not to third-party file hosting sites. If necessary, add the image to a zip or tar archive.

The best reproductions are self-contained scripts with minimal dependencies. If you are using a framework such as plone, Django, or buildout, try to replicate the issue just using Pillow.

from PIL import Image
image = Image.open('test_bitonal_text.tif')
image.save('test_pillow_save.tif', compression='group4')

[jeremydmoore]

[hugovk]

Git bisect points to https://github.com/python-pillow/Pillow/commit/6ead422e91bec1facc644f02bc25a561d0a02383 from PR https://github.com/python-pillow/Pillow/pull/3513.

$ python3 3677.py
Python(31119,0x7fffb3bf1380) malloc: *** mach_vm_map(size=18446744072426508288) failed (error code=3)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
test_pillow_save.tif: Failed to allocate memory for custom tag binary object (-1283043808 elements of 1 bytes each).
Traceback (most recent call last):
  File "3677.py", line 3, in <module>
    image.save('test_pillow_save.tif', compression='group4')
  File "/Users/hugo/github/Pillow/src/PIL/Image.py", line 1988, in save
    save_handler(self, fp, filename)
  File "/Users/hugo/github/Pillow/src/PIL/TiffImagePlugin.py", line 1552, in _save
    e = Image._getencoder(im.mode, 'libtiff', a, im.encoderconfig)
  File "/Users/hugo/github/Pillow/src/PIL/Image.py", line 477, in _getencoder
    return encoder(mode, *args + extra)
RuntimeError: Error setting from dictionary

• OS: macOS High Sierra 10.13.6
• Python: 3.7.2

[jeremydmoore]

If I replace the code in PIL.TiffImagePlugin.py changed by 6ead422 with that from 5.3.0 the save succeeds in 5.4.1

LEFT: TiffImagePlugin.py from 5.4.1 that works RIGHT: TiffImagePlugin.py from 5.3.0 that works

[radarhere]

I have found the problematic tags. The following code runs without error.

from PIL import Image
image = Image.open('test_bitonal_text.tif')
del image.tag_v2[34377]
del image.tag_v2[33723]
del image.tag_v2[700]
image.save('test_pillow_save.tif', compression='group4')

The following code is all that is needed to reproduce the segmentation fault.

from PIL import Image, TiffImagePlugin
TiffImagePlugin.WRITE_LIBTIFF = True
image = Image.new("RGB", (100, 100))
image.save('out.tif', tiffinfo={700: b"a"})

The test suite is able to save byte data for other tag numbers, so the problem is specific to these numbers. The seg fault is occuring at https://github.com/python-pillow/Pillow/blob/d167f9e0bd8a71f7d0a5a1098dc9d51eceb67be2/src/libImaging/TiffDecode.c#L432

[radarhere]

While an error still occurs for this issue, the segfault has been fixed in #4033

[radarhere]

The error now received here is

from PIL import Image
image = Image.open('test_bitonal_text.tif')
image.save('test_pillow_save.tif', compression='group4')

TypeError: an integer is required (got type bytes)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "code.py", line 3, in <module>
    image.save('test_pillow_save.tif', compression='group4')
  File "PIL/Image.py", line 2054, in save
  File "PIL/TiffImagePlugin.py", line 1598, in _save
  File "PIL/Image.py", line 428, in _getencoder
SystemError: <built-in function libtiff_encoder> returned a result with an error set

[zoj613]

Also getting the SystemError: <built-in function libtiff_encoder> returned a result with an error set when daving a tif file through tesserocr. Anyone with a solution yet?

[radarhere]

I'm actually working on a solution at the moment. To be sure that your problem is solved though, in case your error is triggered by a different case than the one in this issue, could we have a simple self-contained example of your situation?

[zoj613]

I'm actually working on a solution at the moment. To be sure that your problem is solved though, in case your error is triggered by a different case than the one in this issue, could we have a simple self-contained example of your situation?

it gets triggered when performing OCR on a tiff PIL image via the tesserocr library. SOmething like:

import tesserocr
from tesserocr import PSM
tesserocr.image_to_text(scanned_tiff_img, psm=PSM.SPARSE_TEXT_OSD)

Then I get SystemError: <built-in function libtiff_encoder> returned a result with an error set. From what I remember from looking at the stacktrace, it jammed when the library tried to save the tiff. My version of PIL is 7.1.2

[hugovk]

@zoj613 Please could you post the full stacktrace?

[NPann]

FWIW, I am getting the same error on the following snippet:

from PIL import Image, TiffImagePlugin                                                                                     
image = Image.open('mytiff.tif')
image.save('test_pillow_save.tif')

with the following stracktrace:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
TypeError: an integer is required (got type bytes)

The above exception was the direct cause of the following exception:

SystemError                               Traceback (most recent call last)
<ipython-input-27-3b2a40a8e19e> in <module>
----> 1 image.save('test_pillow_save.tif')

~/miniconda3/envs/fw/lib/python3.7/site-packages/PIL/Image.py in save(self, fp, format, **params)
   2100 
   2101         try:
-> 2102             save_handler(self, fp, filename)
   2103         finally:
   2104             # do what we can to clean up

~/miniconda3/envs/fw/lib/python3.7/site-packages/PIL/TiffImagePlugin.py in _save(im, fp, filename)
   1614         tags.sort()
   1615         a = (rawmode, compression, _fp, filename, tags, types)
-> 1616         e = Image._getencoder(im.mode, "libtiff", a, im.encoderconfig)
   1617         e.setimage(im.im, (0, 0) + im.size)
   1618         while True:

~/miniconda3/envs/fw/lib/python3.7/site-packages/PIL/Image.py in _getencoder(mode, encoder_name, args, extra)
    429         # get encoder
    430         encoder = getattr(core, encoder_name + "_encoder")
--> 431         return encoder(mode, *args + extra)
    432     except AttributeError:
    433         raise OSError("encoder %s not available" % encoder_name)

SystemError: <built-in function libtiff_encoder> returned a result with an error set

OS: macOS 10.14.6 Python: 3.7.5 Pillow: 7.0.0 and 7.1.2

[radarhere]

I've created PR #4605 to resolve this.

@zoj613 @NPann if you could check and confirm that this also resolves your situations. If it does not, please provide images to demonstrate the problem.

[NPann]

@radarhere, it does resolve my situation. Thanks a lot for figuring this out!