search

python-pillow / Pillow

Python Imaging Library (Fork)
https://python-pillow.org
Other
12.31k stars 2.23k forks source link

FAIL: TestImageFont.test_render_multiline - AssertionError: average pixel value difference 14.2679 > epsilon 0.5000 #1259

Closed jlec closed 8 years ago

jlec commented 9 years ago

Happens with pillow 2.8.2/2.8.1 and pypy, python 2.7, 3.3, 3.4

======================================================================
FAIL: TestImageFont.test_render_multiline
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/pillow-2.8.2/work/Pillow-2.8.2-pypy/Tests/test_imagefont.py", line 148, in test_render_multiline
    self.assert_image_similar(im, target_img, .5)
  File "/var/tmp/portage/dev-python/pillow-2.8.2/work/Pillow-2.8.2-pypy/Tests/helper.py", line 105, in assert_image_similar
    ave_diff, epsilon))
AssertionError:  average pixel value difference 14.2679 > epsilon 0.5000

----------------------------------------------------------------------
jlec commented 9 years ago

I am testing on gentoo linux. I can provide any package version you, just tell me which you need. The test are run with unset DISPLAY. And I am using fontconfig + infinality.

Portage 2.2.20 (python 3.4.3-final-0, default/linux/amd64/13.0, gcc-4.9.2-asneeded, glibc-2.20-r2, 4.0.4-lh x86_64)
=================================================================
System uname: Linux-4.0.4-lh-x86_64-Intel-R-_Core-TM-_i7-2600_CPU_@_3.40GHz-with-gentoo-2.2
KiB Mem:    16399532 total,   1921736 free
KiB Swap:    8388600 total,   8388600 free
Timestamp of repository gentoo: Mon, 08 Jun 2015 05:15:01 +0000
sh bash 4.3_p39
ld GNU ld (Gentoo 2.25 p1.2) 2.25
ccache version 3.2.2 [enabled]
app-shells/bash:          4.3_p39::gentoo
dev-java/java-config:     2.2.0::gentoo
dev-lang/perl:            5.20.2-r1::gentoo
dev-lang/python:          2.7.10::gentoo, 3.3.5-r1::gentoo, 3.4.3::gentoo
dev-util/ccache:          3.2.2::gentoo
dev-util/cmake:           3.2.2::gentoo
dev-util/pkgconfig:       0.28-r3::gentoo
sys-apps/baselayout:      2.2::gentoo
sys-apps/openrc:          0.16.3::gentoo
sys-apps/sandbox:         2.6-r1::gentoo
sys-devel/autoconf:       2.13::gentoo, 2.69-r1::gentoo
sys-devel/automake:       1.11.6-r1::gentoo, 1.12.6::gentoo, 1.13.4::gentoo, 1.14.1::gentoo, 1.15::gentoo
sys-devel/binutils:       2.25-r1::gentoo
sys-devel/gcc:            4.1.2::gentoo, 4.9.2::gentoo
sys-devel/gcc-config:     1.8::gentoo
sys-devel/libtool:        2.4.6-r1::gentoo
sys-devel/make:           4.1-r1::gentoo
sys-kernel/linux-headers: 4.0::gentoo (virtual/os-headers)
sys-libs/glibc:           2.20-r2::gentoo
Repositories:

gentoo
    location: /var/db/gentoo
    sync-type: git
    sync-uri: https://github.com/gentoo/gentoo-portage-rsync-mirror.git
    priority: -1000

betagarden
    location: /local/src/repos/betagarden
    sync-type: git
    sync-uri: https://anongit.gentoo.org/git/proj/betagarden.git
    masters: gentoo
    priority: 50

dummy
    location: /local/src/repos/dummy
    masters: gentoo
    priority: 50

last-hope
    location: /local/src/repos/last-hope
    sync-type: git
    sync-uri: git://github.com/ercpe/lh-overlay.git
    masters: gentoo
    priority: 50

science
    location: /local/src/repos/science
    sync-type: git
    sync-uri: git://github.com/gentoo-science/sci.git
    masters: gentoo
    priority: 50

gentoo-cvs
    location: /local/home/justin/tree
    masters: gentoo
    priority: 1000

ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -ftracer   -march=native -frecord-gcc-switches -g -fdiagnostics-color=always -Wimplicit-function-declaration -Wformat -Wformat-security"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/gnupg/qualified.txt /var/lib/hsqldb /var/spool/munin-async/.ssh /var/spool/torque"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -pipe -ftracer   -march=native -frecord-gcc-switches -g -fdiagnostics-color=always -Wenum-compare -Wformat -Wformat-security"
DISTDIR="/var/cache/distfiles"
EMERGE_DEFAULT_OPTS=" -vt --keep-going --autounmask-write --autounmask --quiet-build=n --autounmask-unrestricted-atoms=y"
FCFLAGS="-O2 -pipe -ftracer   -march=native -frecord-gcc-switches -g -fdiagnostics-color=always"
FEATURES="assume-digests binpkg-logs buildsyspkg ccache cgroup collision-protect compressdebug distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news noinfo parallel-fetch parallel-install preserve-libs protect-owned sandbox sfperms sign split-log splitdebug strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe -ftracer   -march=native -frecord-gcc-switches -g -fdiagnostics-color=always"
GENTOO_MIRRORS="    /mnt/tmpfs/     http://gentoo.j-schmitz.net/mirror/     ftp://sunsite.informatik.rwth-aachen.de/pub/Linux/gentoo    ftp://ftp.tu-clausthal.de/pub/linux/gentoo/"
LANG="en_GB.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,-O1 -Wl,--hash-style=gnu -Wl,--sort-common"
MAKEOPTS="-j12 -l8"
PKGDIR="/var/cache/packages"
PORTAGE_COMPRESS="xz"
PORTAGE_COMPRESS_FLAGS="-z -e -9 -v --threads=4"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_EXTRA_OPTS="--omit-dir-times"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
USE="X acl alsa amd64 avx bash-completion berkdb branding bzip2 cairo caps cli cracklib crypt cups cxx dbus dri fortran gdbm gif gmp gstreamer gtk gtk3 iconv icu ipv6 jpeg jpeg2k ldap logrotate mmx mmxext modules multilib ncurses network-cron nls nptl nsplugin numa opengl openmp pam pcre pgo png pulseaudio qt3support readline session smp sse sse2 sse2_4way sse2check sse3 sse4 sse41 sse4_1 sse4_2 sse4a sse5 ssl ssse3 startup-notification tcpd threads tiff truetype unicode vaapi vdpau xattr xinerama zlib" ABI_X86="64" ALSA_CARDS="hda-intel" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" DRACUT_MODULES="plymouth" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="efi-32 efi-64" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer pdfimport scripting-javascript wiki-publisher nlpsolver" LINGUAS="en en_GB en_US" NGINX_MODULES_HTTP="access auth_basic autoindex browser charset empty_gif fastcgi geo gzip limit_conn limit_req map memcached proxy referer rewrite scgi split_clients ssi upstream_ip_hash userid uwsgi stub_status" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3 python3_4 pypy" QEMU_SOFTMMU_TARGETS="i386 x86_64 arm armeb" QEMU_USER_TARGETS="i386 x86_64 arm armeb" RUBY_TARGETS="ruby20" USERLAND="GNU" VIDEO_CARDS="nvidia" XFCE_PLUGINS="clock trash power" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
USE_PYTHON="2.7 3.3"
Unset:  CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND
hugovk commented 9 years ago

(Tip: place three backticks (```) before and after quoted code or output for monospace formatting and to avoid GitHub adding odd titles. I've edited these messages :)

wiredfool commented 9 years ago

A difference of ~14 would be reasonable with a change in kerning or font shapes, and since infinality appears to be a patch set on freetype that affects the rendering, I'd expect that pillow is still functioning properly.

If you can eyeball the output file from that test and it looks correct, I'd say that appropriate responses are to modify the gentoo patch set when building against infinality by either: bump up the epsilon in the test, or modify the target image with an updated version.

If it's not correct, then it's likely an interaction with the freetype patches.

jlec commented 9 years ago

I will see what the image look like, and try how different settings for fontconfig change the situation.

jlec commented 9 years ago

@wiredfool I cannot find any fontconfig option which changes the result. Here a version of what I am rendering. multiline_text_new

jlec commented 9 years ago

Still failing with 3.0.0

======================================================================
FAIL: TestImageFont.test_multiline_spacing
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/pillow-3.0.0/work/Pillow-3.0.0-pypy/Tests/test_imagefont.py", line 213, in test_multiline_spacing
    self.assert_image_similar(im, target_img, .5)
  File "/var/tmp/portage/dev-python/pillow-3.0.0/work/Pillow-3.0.0-pypy/Tests/helper.py", line 100, in assert_image_similar
    ave_diff, epsilon))
AssertionError:  average pixel value difference 14.2679 > epsilon 0.5000
-------------------- >> begin captured logging << --------------------
PIL.PngImagePlugin: DEBUG: STREAM IHDR 16 13
PIL.PngImagePlugin: DEBUG: STREAM IDAT 41 2787
--------------------- >> end captured logging << ---------------------

----------------------------------------------------------------------
Ran 445 tests in 15.576s
wiredfool commented 9 years ago

Right, infinality is still messing with the kerning. I'd bet that it's the same problem that's hitting you on #1456, as kerning could affect that one as well.

This is probably a gentoo packaging issue.

jlec commented 9 years ago 
======================================================================
FAIL: TestImageFont.test_textsize_equal
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/pillow-3.0.0/work/Pillow-3.0.0-pypy/Tests/test_imagefont.py", line 127, in test_textsize_equal
    self.assert_image_similar(im, target_img, .5)
  File "/var/tmp/portage/dev-python/pillow-3.0.0/work/Pillow-3.0.0-pypy/Tests/helper.py", line 100, in assert_image_similar
    ave_diff, epsilon))
AssertionError:  average pixel value difference 2.4842 > epsilon 0.5000
-------------------- >> begin captured logging << --------------------
PIL.PngImagePlugin: DEBUG: STREAM IHDR 16 13
PIL.PngImagePlugin: DEBUG: STREAM IDAT 41 1463
--------------------- >> end captured logging << ---------------------

----------------------------------------------------------------------
Ran 453 tests in 14.974s
jlec commented 9 years ago

The problem could be that we are running the test in a fake X env which has a small (1280x1024x24) screen.

wiredfool commented 9 years ago

That's unlikely to affect things much if it's stable across such a wide range of other environments without the infinality+fontconfig patches.