PyAr / CDPedia

CDPedia is a project to make the Wikipedia accesable offline
34 stars 15 forks source link

Support PIL crashing on the image #393

Closed facundobatista closed 3 years ago

facundobatista commented 3 years ago

I experienced several different crashes from PIL when working with the image; we should not crash, log in debug that PIL had a problem (showing the error type and description), and go on.

Examples of what I saw:

2021-06-16 22:03:21,835  src.utiles           ERROR    Crashed while processing ('http://upload.wikimedia.org/wikipedia/commons/thumb/c/c2/Pt-sns3.png/100px-Pt-sns3.png', '/massive/cdpedia/dump/images/commons/thumb/c/c2/100px-Pt-sns3.png'): UnidentifiedImageError("cannot identify image file '/massive/cdpedia/dump/images/commons/thumb/c/c2/100px-Pt-sns3.png'")
Traceback (most recent call last):
  File "/massive/cdpedia/CDPedia/src/utiles.py", line 115, in process
    self.func(payload)
  File "/massive/cdpedia/CDPedia/src/images/download.py", line 119, in download
    optimize_image(fullpath)
  File "/massive/cdpedia/CDPedia/src/images/download.py", line 58, in optimize_image
    with Image.open(img_path) as img:
  File "/home/facundo/.local/share/fades/3cc1bd5a-18e6-4c59-8090-50b81bcfbd88/lib/python3.8/site-packages/PIL/Image.py", line 2967, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file '/massive/cdpedia/dump/images/commons/thumb/c/c2/100px-Pt-sns3.png'

...

2021-06-17 03:05:40,664  src.utiles           ERROR    Crashed while processing ('http://upload.wikimedia.org/wikipedia/commons/thumb/5/5d/Labomar.tif/lossless-page1-220px-Labomar.tif.png', '/massive/cdpedia/dump/images/commons/thumb/5/5d/lossless-page1-220px-Labomar.tif.png'): ValueError('Decompressed Data Too Large')
Traceback (most recent call last):
  File "/massive/cdpedia/CDPedia/src/utiles.py", line 115, in process
    self.func(payload)
  File "/massive/cdpedia/CDPedia/src/images/download.py", line 119, in download
    optimize_image(fullpath)
  File "/massive/cdpedia/CDPedia/src/images/download.py", line 58, in optimize_image
    with Image.open(img_path) as img:
  File "/home/facundo/.local/share/fades/3cc1bd5a-18e6-4c59-8090-50b81bcfbd88/lib/python3.8/site-packages/PIL/Image.py", line 2953, in open
    im = _open_core(fp, filename, prefix, formats)
  File "/home/facundo/.local/share/fades/3cc1bd5a-18e6-4c59-8090-50b81bcfbd88/lib/python3.8/site-packages/PIL/Image.py", line 2939, in _open_core
    im = factory(fp, filename)
  File "/home/facundo/.local/share/fades/3cc1bd5a-18e6-4c59-8090-50b81bcfbd88/lib/python3.8/site-packages/PIL/ImageFile.py", line 121, in __init__
    self._open()
  File "/home/facundo/.local/share/fades/3cc1bd5a-18e6-4c59-8090-50b81bcfbd88/lib/python3.8/site-packages/PIL/PngImagePlugin.py", line 694, in _open
    s = self.png.call(cid, pos, length)
  File "/home/facundo/.local/share/fades/3cc1bd5a-18e6-4c59-8090-50b81bcfbd88/lib/python3.8/site-packages/PIL/PngImagePlugin.py", line 187, in call
    return getattr(self, "chunk_" + cid.decode("ascii"))(pos, length)
  File "/home/facundo/.local/share/fades/3cc1bd5a-18e6-4c59-8090-50b81bcfbd88/lib/python3.8/site-packages/PIL/PngImagePlugin.py", line 545, in chunk_zTXt
    v = _safe_zlib_decompress(v[1:])
  File "/home/facundo/.local/share/fades/3cc1bd5a-18e6-4c59-8090-50b81bcfbd88/lib/python3.8/site-packages/PIL/PngImagePlugin.py", line 133, in _safe_zlib_decompress
    raise ValueError("Decompressed Data Too Large")
ValueError: Decompressed Data Too Large