search

kornelski / pngquant

Lossy PNG compressor — pngquant command based on libimagequant library
https://pngquant.org
Other
5.25k stars 487 forks source link

Input reduced to 1-color image #346

Closed thomascube closed 4 years ago

thomascube commented 5 years ago

Using pngquant 2.12.5 on CentOS 7 (installed via epel repo), the resulting image is reduced to a single color which is certainly not the desired result. Neither the --speednor the --nofs options do have any effect on the result. Only when using the --posterize option, the result is the expected 8-bit palette image. But here again, only with posterize values 3 and 4. The same 1-color image results with --posterize [0,1,2]. Tested with different PNG images with and without alpha colors. It also only happens with the 2.12.5 version on CentOS but works fine with pngquant 2.12.5 on Mac OS (installed via homebrew).

Here's some verbose output that might help you investigate the issue:

Version

$ pngquant -h
pngquant, 2.12.5 (July 2019), by Kornel Lesinski, Greg Roelofs.
   Compiled with OpenMP (multicore support).
   Color profiles are supported via Little CMS. Using libpng 1.5.13.

WARNING: Your version of libpng is old and has buggy support for custom chunks.
Please recompile pngquant with the current version of libpng (1.6 or later).

...

Call without posterize option

$ pngquant --speed 9 -f -v ws_flowers.png 
ws_flowers.png:
  read 652KB file
  made histogram...50265 colors found
  eliminated opaque tRNS-chunk entries...1 entry transparent
  writing 1-color image as ws_flowers_na-fs8.png
Quantized 1 image.

Call with posterize option

$ pngquant --speed 9 --posterize 3 -f -v -o quant.png ws_flowers.png 
ws_flowers.png:
  read 652KB file
  made histogram...4663 colors found
  eliminated opaque tRNS-chunk entries...8 entries transparent
  writing 256-color image as quant.png
Quantized 1 image.
zvezdochiot commented 5 years ago

@thomascube say:

Using pngquant 2.12.5 on CentOS 7 (installed via epel repo)

Wrong address. You to the CentOS 7 maintainers.

kornelski commented 5 years ago

Can you share the file you're converting? It's certainly very unexpected that 50K colors are squashed down to 1.

thomascube commented 5 years ago

I'm mostly using PNGs saved with Imagemagick but it seems to happen with any random PNG. Here are two examples: sample stormy

zvezdochiot commented 5 years ago

@thomascube say:

Here are two examples:

$ pngquant --speed 9 -f -v 67459055-f4128780-f637-11e9-8d51-c9ae2e375d11.png 
67459055-f4128780-f637-11e9-8d51-c9ae2e375d11.png:
  read 656KB file
  made histogram...29094 colors found
  eliminated opaque tRNS-chunk entries...0 entries transparent
  writing 256-color image as 67459055-f4128780-f637-11e9-8d51-c9ae2e375d11-fs8.png
Quantized 1 image.
$ pngquant --speed 9 -f -v 67459079-0096e000-f638-11e9-83be-7af550a84b40.png 
67459079-0096e000-f638-11e9-83be-7af550a84b40.png:
  read 672KB file
  passing sRGB tag from the input
  made histogram...27661 colors found
  eliminated opaque tRNS-chunk entries...0 entries transparent
  writing 256-color image as 67459079-0096e000-f638-11e9-83be-7af550a84b40-fs8.png
Quantized 1 image.

67459055-f4128780-f637-11e9-8d51-c9ae2e375d11-fs8.png 67459079-0096e000-f638-11e9-83be-7af550a84b40-fs8.png

joejordanbrown commented 5 years ago

@thomascube @kornelski

I've been working on debugging this issue over the past week, from what I can tell this only affects pngquant from EPEL repo on CentOS 7, pngquant-2.12.5-1.el7.x86_64.rpm.

It seems to be a problem with their build process of pngquant using gcc version 4.8.5 which is the default version of gcc on CentosOS 7.

I have a repo to showcase this issue here https://github.com/joejordanbrown/pngquant-epel-package-issue

I've created a bug report with RedHat to have the package rebuilt with a newer version of gcc and also requested for them to use a newer version of libpng. They previously compiled pngquant-2.12.5-1 with libpng-1.5.13 which is latest version in the EPEL repo, so will need to be compiled from source.

https://bugzilla.redhat.com/show_bug.cgi?id=1765388

zvezdochiot commented 5 years ago

@joejordanbrown say:

I've created a bug report with RedHat to have the package rebuilt with a newer version of gcc

It is much wiser to compile libimagequant0 and pngquant with the smaller version of gcc and libpng12. This will ensure both compatibility and uptime.

PS:

You can also use the alien system:

zvezdochiot commented 5 years ago

@thomascube say:

Using pngquant 2.12.5

Where do you get this?

joejordanbrown commented 5 years ago

@joejordanbrown say:

I've created a bug report with RedHat to have the package rebuilt with a newer version of gcc

It is much wiser to compile libimagequant0 and pngquant with the smaller version of gcc and libpng12. This will ensure both compatibility and uptime.

PS:

You can also use the alien system:

...

I'm not sure if this is the right idea to compile with an older version of gcc unless it's explicitly stated in the installation docs.

Are you aware of what version of gcc this software is tested with?

Also pngquant shows a warning to use 1.6 and later.

WARNING: Your version of libpng is old and has buggy support for custom chunks. Please recompile pngquant with the current version of libpng (1.6 or later).

So how come you recommend a lower version, if the software states differently?

For our testing I created a perl script that uses ImageMagick ^v7 to compare image difference of the input and output of pngquant, this is a pretty accurate method of testing the lib.

You can find it here https://github.com/joejordanbrown/pngquant-epel-package-issue/blob/master/scripts/diff-image.pl this could be easily used to test 10000s of images with each version to find possible issues.

@thomascube say:

Using pngquant 2.12.5

Where do you get this?

I think you might have quickly glanced at the releases page and seen 2.12.1 at the top but missed the other releases and tags above. I know it's not clear, see highlighted in red.

2.12.5 is the latest version, the github releases page states this.

The real latest release is shown on pngquant.org website, and probably not here.

I don't update this GitHub page often. Check the website instead.

image

image

image

zvezdochiot commented 5 years ago

@joejordanbrown say:

I'm not sure if this is the right idea to compile with an older version of gcc unless it's explicitly stated in the installation docs.

pngquant-2.12.5$ gcc --version
gcc (Debian 4.7.2-5) 4.7.2
pngquant-2.12.5$ ./pngquant 
pngquant, 2.12.5 (July 2019), by Kornel Lesinski, Greg Roelofs.
   Color profiles are supported via Little CMS. Using libpng 1.2.49.
pngquant-2.12.5$ ldd pngquant 
    linux-gate.so.1 =>  (0xb7762000)
    libimagequant.so.0 => /usr/lib/i386-linux-gnu/libimagequant.so.0 (0xb7701000)
    libpng12.so.0 => /lib/i386-linux-gnu/libpng12.so.0 (0xb76d7000)
    libz.so.1 => /lib/i386-linux-gnu/libz.so.1 (0xb76bd000)
    liblcms2.so.2 => /usr/lib/i386-linux-gnu/liblcms2.so.2 (0xb7670000)
    libm.so.6 => /lib/i386-linux-gnu/i686/cmov/libm.so.6 (0xb764a000)
    libc.so.6 => /lib/i386-linux-gnu/i686/cmov/libc.so.6 (0xb74e2000)
    /lib/ld-linux.so.2 (0xb7765000)
pngquant-2.12.5$ ./pngquant -v 67459055-f4128780-f637-11e9-8d51-c9ae2e375d11.png 
67459055-f4128780-f637-11e9-8d51-c9ae2e375d11.png:
  read 656KB file
  made histogram...51308 colors found
  selecting colors...11%
  selecting colors...22%
  selecting colors...33%
  selecting colors...100%
  moving colormap towards local minimum
  eliminated opaque tRNS-chunk entries...0 entries transparent
  mapped image to new colors...MSE=2.414 (Q=91)
  writing 256-color image as 67459055-f4128780-f637-11e9-8d51-c9ae2e375d11-fs8.png
Quantized 1 image.
pngquant-2.12.5$ ./pngquant -v 67459079-0096e000-f638-11e9-83be-7af550a84b40.png 
67459079-0096e000-f638-11e9-83be-7af550a84b40.png:
  read 672KB file
  passing sRGB tag from the input
  made histogram...81573 colors found
  selecting colors...11%
  selecting colors...77%
  selecting colors...88%
  selecting colors...100%
  moving colormap towards local minimum
  eliminated opaque tRNS-chunk entries...0 entries transparent
  mapped image to new colors...MSE=2.367 (Q=91)
  writing 256-color image as 67459079-0096e000-f638-11e9-83be-7af550a84b40-fs8.png
Quantized 1 image.

Images are presented above.

kornelski commented 5 years ago

Your problematic version is compiled with OpenMP, so this is probably the cause of it. I've tried to be very conservative with multithreading in C, but apparently that's still not enough.

joejordanbrown commented 5 years ago

@zvezdochiot

I see you tested on Debian and without all the options for OpenMP. Our configure command is ./configure --with-openmp --with-lcms2 --prefix=/usr CC=${gcc_path}.

I'm only testing now on RedHat releases because that's only where I've managed to replicate the issue.

Running our tests using gcc 4.9.2 from devtoolset-3-gcc-4.9.2-6.el7.x86_64.rpm and this works fine and resolves the 1/2-color issue.

Thanks for taking the time with helping to debug this though, much appreciated 😁

@kornelski

Your problematic version is compiled with OpenMP, so this is probably the cause of it. I've tried to be very conservative with multithreading in C, but apparently that's still not enough.

Yes, thanks for that, OpenMP is the issue. I tested with that flag off and gcc 4.8.5 compiles a functioning pngquant.

I searched to see if there were known issues with that version of gcc and OpenMP, and there seem to be a number of bugs around that version of gcc and OpenMP.

When compiling with gcc 4.8.5 and OpenMP 3.1 I notice this warning.

mediancut.c: In function 'mediancut':
mediancut.c:346:0: warning: ignoring #pragma omp taskgroup [-Wunknown-pragmas]
         #pragma omp taskgroup
 ^
mediancut.c:394:0: warning: ignoring #pragma omp taskgroup [-Wunknown-pragmas]
             #pragma omp taskgroup
 ^

At first, I wasn't sure if it was relevant, but then I realised omp is short for OpenMP. Found it mentioned in OpenMP API Syntax Reference Guide, for example, https://www.openmp.org/wp-content/uploads/OpenMPRef-5.0-0519-web.pdf.

@kornelski

From your perspective what do you think the best configuration options for a distributed package of pngquant should be?

My choice right now is ./configure --with-openmp --with-lcms2, if you think it could be improved let me know and I'll update the RedHat ticket to release an updated package using a none buggy version of gcc.

kornelski commented 5 years ago

./configure --with-openmp --with-lcms2 would be ideal if not the concurrency bugs :) ./configure --with-lcms2 is the second best.

I've been using gcc 7 to develop this, so I would not recommend touching OpenMP on anything older.

kornelski commented 5 years ago

Did you get warnings about also ignoring omp task? If the compiler supports task, but not taskgroup, that would explain the why the code has race conditions.

I'm surprised that an OpenMP-supporting compiler doesn't treat unknown omp pragma as a hard error.

I could probably remove all uses of OpenMP tasks, because they have very minor impact. Maybe I'll redo them in rayon someday :)

kornelski commented 4 years ago

Dropped use of omp task.