search

azaghal / pydenticon

Pydenticon is a small utility library that can be used for deterministically generating identicons based on the hash of provided data.
BSD 3-Clause "New" or "Revised" License
66 stars 10 forks source link

self-tests fail in test_generate_format and following on big endian architecture with "tile cannot extend outside image" #10

Open okurz opened 5 years ago

okurz commented 5 years ago

Observation

the openSUSE build service (OBS) shows reproducible test failures on big endian architectures, e.g. ppc64 and s390x, while little endian is fine:

https://build.opensuse.org/package/show/devel:languages:python/python-pydenticon shows the "failed" builds and tests.

In detail from https://build.opensuse.org/package/live_build_log/devel:languages:python/python-pydenticon/openSUSE_Factory_PowerPC/ppc64:

[   81s] ======================================================================
[   81s] ERROR: test_generate_format (tests.test_pydenticon.GeneratorTest)
[   81s] ----------------------------------------------------------------------
[   81s] Traceback (most recent call last):
[   81s]   File "/home/abuild/rpmbuild/BUILD/pydenticon-0.3.1/tests/test_pydenticon.py", line 236, in test_generate_format
[   81s]     raw_image = generator.generate(data, 200, 200, output_format="png")
[   81s]   File "/home/abuild/rpmbuild/BUILD/pydenticon-0.3.1/pydenticon/__init__.py", line 341, in generate
[   81s]     return self._generate_image(matrix, width, height, padding, foreground, background, output_format)
[   81s]   File "/home/abuild/rpmbuild/BUILD/pydenticon-0.3.1/pydenticon/__init__.py", line 254, in _generate_image
[   81s]     image.save(stream, format=image_format, optimize=True)
[   81s]   File "/usr/lib64/python2.7/site-packages/PIL/Image.py", line 2007, in save
[   81s]     save_handler(self, fp, filename)
[   81s]   File "/usr/lib64/python2.7/site-packages/PIL/PngImagePlugin.py", line 896, in _save
[   81s]     [("zip", (0, 0)+im.size, 0, rawmode)])
[   81s]   File "/usr/lib64/python2.7/site-packages/PIL/ImageFile.py", line 496, in _save
[   81s]     e.setimage(im.im, b)
[   81s] SystemError: tile cannot extend outside image
[   81s] 
[   81s] ======================================================================
[   81s] ERROR: test_generate_image_basics (tests.test_pydenticon.GeneratorTest)
[   81s] ----------------------------------------------------------------------
[   81s] Traceback (most recent call last):
[   81s]   File "/home/abuild/rpmbuild/BUILD/pydenticon-0.3.1/tests/test_pydenticon.py", line 182, in test_generate_image_basics
[   81s]     raw_image = generator._generate_image(matrix, width, height, padding, foreground, background, "png")
[   81s]   File "/home/abuild/rpmbuild/BUILD/pydenticon-0.3.1/pydenticon/__init__.py", line 254, in _generate_image
[   81s]     image.save(stream, format=image_format, optimize=True)
[   81s]   File "/usr/lib64/python2.7/site-packages/PIL/Image.py", line 2007, in save
[   81s]     save_handler(self, fp, filename)
[   81s]   File "/usr/lib64/python2.7/site-packages/PIL/PngImagePlugin.py", line 896, in _save
[   81s]     [("zip", (0, 0)+im.size, 0, rawmode)])
[   81s]   File "/usr/lib64/python2.7/site-packages/PIL/ImageFile.py", line 496, in _save
[   81s]     e.setimage(im.im, b)
[   81s] SystemError: tile cannot extend outside image
[   81s] 
[   81s] ======================================================================
[   81s] ERROR: test_generate_image_compare (tests.test_pydenticon.GeneratorTest)
[   81s] ----------------------------------------------------------------------
[   81s] Traceback (most recent call last):
[   81s]   File "/home/abuild/rpmbuild/BUILD/pydenticon-0.3.1/tests/test_pydenticon.py", line 383, in test_generate_image_compare
[   81s]     raw_image = generator.generate("test1", width, height, padding=padding)
[   81s]   File "/home/abuild/rpmbuild/BUILD/pydenticon-0.3.1/pydenticon/__init__.py", line 341, in generate
[   81s]     return self._generate_image(matrix, width, height, padding, foreground, background, output_format)
[   81s]   File "/home/abuild/rpmbuild/BUILD/pydenticon-0.3.1/pydenticon/__init__.py", line 254, in _generate_image
[   81s]     image.save(stream, format=image_format, optimize=True)
[   81s]   File "/usr/lib64/python2.7/site-packages/PIL/Image.py", line 2007, in save
[   81s]     save_handler(self, fp, filename)
[   81s]   File "/usr/lib64/python2.7/site-packages/PIL/PngImagePlugin.py", line 896, in _save
[   81s]     [("zip", (0, 0)+im.size, 0, rawmode)])
[   81s]   File "/usr/lib64/python2.7/site-packages/PIL/ImageFile.py", line 496, in _save
[   81s]     e.setimage(im.im, b)
[   81s] SystemError: tile cannot extend outside image

Reproducible

yes, on ppc64 and s390x every time. Should be reproducible in a VM with cross-virtualization.

Problem

I investigated if I can find any similar report for PIL or Pillow. What I have found is mentions of endianess on https://github.com/python-pillow/Pillow/search?q=endian&type=Issues but no specifically related report, the following might be helpful though:

Also, https://build.opensuse.org/package/show/devel:languages:python/python-Pillow shows the build and test results for Pillow which are mostly successful however internally the tests also fail, e.g. on s390x

eirnym commented 5 years ago

@okurz Is this problem of pydenticon project or it's the matter to update Pillow version?

I believe if you report here, underlying problem with Pillow Is solved and you're just about to report to update dependency.

azaghal commented 5 years ago

Can't guarantee it, but most likely it's just a dependency issue. At least Pydenticon does not try to use anything super-exotic that might (directly) depend on arch specifics.

Could you let me know what version of Pillow seems to cause the issue, and whether you can reproduce the issue with latest Pillow?

azaghal commented 5 years ago

I mean, the issue most likely within Pillow, and not Pydenticon itself :)

azaghal commented 5 years ago

Any recommendations on what I could get up and running for testing this locally or on some remote machine?

eirnym commented 5 years ago

@azaghal I've found a SO question

okurz commented 5 years ago

I did not check this but it should be possible to reproduce with e.g. qemu-system-ppc64. On openSUSE that is part of the package "qemu-ppc". Similar for s390x. For other distributions I suggest to look on e.g. https://pkgs.org/download/qemu-system-s390x

Everyone can create an account on the Open Build Service https://build.opensuse.org so you can even try to reproduce the problem in there by branching the package and trying around.

Pillow is used in version 6.0.0 which is the latest release according to https://github.com/python-pillow/Pillow/releases from 2019-04-02 . I agree that the problem could very well just be within Pillow and admittedly the tests of Pillow on the same architectures mentioned above fail already quite severly as can be seen in https://build.opensuse.org/package/live_build_log/devel:languages:python/python-Pillow/openSUSE_Factory_zSystems/s390x . The tests do not even run till the end because of a SEGFAULT :D

If you prefer we can close the issue here again and leave it to Pillow to fix this. I can also report the issue there if preferred.

eirnym commented 5 years ago

One of the best solutions is to create an issue for Pillow and to link it here. The issue here can wait open until upstream issue will be fixed.

okurz commented 5 years ago

Agreed. I added another comment on https://github.com/python-pillow/Pillow/issues/1204#issuecomment-500704056 and referenced this issue there.

azaghal commented 5 years ago

I actually found a solution for running MIPS locally that's a bit more up-to-date (should be big endian, I think - https://aircrack-ng.blogspot.com/2018/10/to-be-or-not-to-be-using-qemu-to-run.html). Might end-up trying to figure out exactly what exact library is causing the issue in testing :)

I wonder if package works at all correctly on big endian arches, though?

okurz commented 5 years ago

the machines building on OBS (open build service, https://build.opensuse.org) use native virtualization though, i.e. s390x jobs are running on s390x machines.

azaghal commented 5 years ago

Getting slightly OT, but can I get direct logins into those machines as well or only trigger automated builds?

okurz commented 5 years ago

No, direct login is not possible