python-needle / needle

Automated tests for your CSS.
https://needle.readthedocs.io/
Other
590 stars 50 forks source link

Empty diff file when using PerceptualDiff #81

Open timdiels opened 4 years ago

timdiels commented 4 years ago

This is actually a duplicate of #61 but it got closed by the reporter.

I'm using the latest perceptualdiff, needle 0.5.0, python 3.7 (and pytest-needle 0.3.11).

When running a simple test and the new image does not match the baseline:

Traceback (most recent call last):
  File ".../test.py", line 174, in test
    needle.assert_screenshot('something')
  File ".../lib/python3.7/site-packages/pytest_needle/driver.py", line 322, in assert_screenshot
    self.engine.assertSameFiles(fresh_image_file, baseline_image, threshold)
  File ".../lib/python3.7/site-packages/needle/engines/perceptualdiff_engine.py", line 38, in assertSameFiles
    Image.open(diff_ppm).save(diff_png)
  File ".../lib/python3.7/site-packages/PIL/Image.py", line 2818, in open
    raise IOError("cannot identify image file %r" % (filename if filename else fp))
OSError: cannot identify image file '.../screenshots/something.diff.ppm'

something.diff.ppm turns out to be an empty file. When running perceptualdiff manually:

$ perceptualdiff baseline/something.png something.png --output some.ppm
FAIL: Images are visibly different
86451 pixels are different
Failed to save to 'some.ppm'

It shows an error, but as it turns out, this does work:

$ perceptualdiff baseline/something.png something.png --output some.png
FAIL: Images are visibly different
86451 pixels are different
Wrote difference image to some.png

perceptualdiff uses FreeImage since 1.0.2 (and it still does):

1.0.2 - [jt] Converted the loading and saving routines to use FreeImage

Even though FreeImage supports PPM, for some reason it doesn't work on my machine. Oddly enough gimp can write PPM and my image viewer can view it too. Anyway, all this can be avoided by requesting a png from perceptualdiff instead of a ppm and then converting it to png; and update the docs to require at least perceptualdiff >= 1.0.2.