Get Killed message for 6.3 MB jpg image

[wpbullet]

I am writing a tutorial on how to batch optimize images using cli with guetzli. The test image I am using for consistency compared to my other tutorials is https://www.cfa.harvard.edu/sites/www.cfa.harvard.edu/files/images/pr/2015-02/1/hires.jpg

I run this command on a 1 GB VPS guetzli --verbose --nomemlimit hires.jpg hires-guetzli.jpg

After a few minutes the only output is Killed There is no output image either, smaller images work fine like the bees sample image in the README here.

I tried limiting the memory usage to 128 and got the same result

guetzli --verbose --memlimit 128 hires.jpg hires-guetzli.jpg

[rogierlommers]

I'm also having the same error / behaviour every now and then. Anyone some suggestions how to work-around?

[khavishbhundoo]

Guetzli uses huge amount of memory at this point.Killed means you ran out of ram

Note: Guetzli uses a large amount of memory. You should provide 300MB of memory per 1MPix of the input image.

Note: Guetzli uses a significant amount of CPU time. You should count on using about 1 minute of CPU per 1 MPix of input image.

https://github.com/google/guetzli/issues/50

[wpbullet]

Thanks @khavishbhundoo using a VM with more RAM did resolve the issue. The image was reduced to 2.7 MB, the same image was reduced to about 1.5 MB with jpeg-recompress (which uses mozjpeg) using the smallfry method (outlined here).

Are there plans to make guetzli more RAM friendly?

[khavishbhundoo]

Thanks @khavishbhundoo using a VM with more RAM did resolve the issue. The image was reduced to 2.7 MB, the same image was reduced to about 1.5 MB with jpeg-recompress (which uses mozjpeg) using the smallfry method (outlined here).

You are misunderstanding how guetzli work.In your example you are using setting the quality to 60. Guetzli uses feedback from butteraugli to create JPEG with no noticeable quality loss.Overall the quality of the JPEG generated is at quality 95 but you can decrease it by up to 84.However guetzli will not let you go below 84 as artifacts would be present in the image at that quality.

Are there plans to make guetzli more RAM friendly?

@robryk might give you an update on this

[wpbullet]

@khavishbhundoo I just had high hopes for guetzli. Good to know below 84 isn't possible

[khavishbhundoo]

@wpbullet Guetzli generates smaller JPEG at great quality that's pretty awesome in my book.At quality 60 you are sacrificing a lot of quality.If you want maximum saving try using both try using both guetzli and jpeg-recompress at quality 84.

[wpbullet]

@khavishbhundoo my understanding of jpeg-recompress is that it does test compressions at different quality ranges and does its best to make sure the noticeable quality difference is minimal before accepting a lower quality. Does guetzli work the same way? I assume yes based on the verbose output but would appreciate any clarification you can offer.

[khavishbhundoo]

@wpbullet guetzli uses butteraugli at its core to ensure that the noticeable quality according to the human eye is minimal.Other tools use PSNR or SSIM.

Butteraugli is a project that estimates the psychovisual similarity of two images. It gives a score for the images that is reliable in the domain of barely noticeable differences. Butteraugli not only gives a scalar score, but also computes a spatial map of the level of differences.

One of the main motivations for this project is the statistical differences in location and density of different color receptors, particularly the low density of blue cones in the fovea. Another motivation comes from more accurate modeling of ganglion cells, particularly the frequency space inhibition.

Read the guetzli paper to know how it works

[DeeDeeG]

@wpbullet I think basically the answer to your question is yes.

As @khavishbhundoo mentioned, Guetzli optimizes based on the Butteraugli metric. That paper is very informative also.

Here is a writeup of how Guetzli decides what output compressed image to use:

There is always a Quality value fed into Guetzli; if you don't specify a quality, it defaults to 95. There is a target Butteraugli score associated with each input Quality value, from 84 (min) to 110 (max) Q. Target scores are listed here: quality.cc. Guetzli makes several attempts at an optimized output, in two main stages... and then among the several atempts, picks whichever is evaluated as being closest to the target Butteraugli score.

(However it won't pick anything worse than 97% of the target score. If the attempt that got closest to the target was worse than 97% of the target quality, Guetzli would choose the next closest attempt above 97% of the target quality.)

(My opinions/interpretations: Guetzli pretty much tries to match and one-up libjpeg-turbo, for two reasons: to prove a point, that there can be something better than a widely accepted status quo (libjpeg and forks)... And so that the quality parameter will work more or less as people expect it to, coming from a libjpeg-turbo workflow. (Other reasons might include that this provides a sane default... And that it generally makes directly comparing results between Guetzli and libjpeg-turbo much easier.)

Butteraugli is a slow, resource intensive, but probably rather effective algorithm to see if two images look similar, approximating how much a human eye could tell the difference between the two. Who knows if it works as it should, but the output images look fine to me.

Guetzli furthermore does some things that allow it to discard data and reduce filesize, without ending up being too bad by Butteraugli's judgment.

Overall this is a very intensive way to solve the problem, but the results are actually pretty good imo. Input images larger than a certain number of megapixels... It pretty much isn't usable for them unless some optimization is done.

So its use case is more specialized than other encoders IMO, and in many edge cases the other encoders are still needed or easier or straight-up better I reckon, but if it's appropriate to the job at hand, Guetzli does its job well.)

[DeeDeeG]

Note that most encoders do chroma downsampling by default, meaning they save only half or quarter-resolution data for color (and full resolution for brightness). This can cause weird color degradations, but I admit they are subtle. For a more direct comparison, try other encoders with chroma sampling disabled. In Mozjpeg on the commandline, this is done with the -sample 1x1 flag. With jpeg-recompress, this is done with --subsample disable.

(For certain images chroma subsampling doesn't adversely affect the image very noticeably, because color transitions aren't very abrupt, and in these cases subsampling encoders tend to win out. (Sometimes subsampling makes the colors very dull in parts of the image, or changes the color to domething inappropriate from a neighboring pixel, and can look very off. Other times you hardly notice. I suspect your demo image, with its high-res/large, gradual/gentle gradients, is a good candidate for chroma subsampling.) Chroma sampling is a very aggressive compression technique; If you can get away with it, it has a big impact on reducing filesize. Guetzli doesn't do chroma subsampling at the moment, it just saves full-resolution color, but there is an issue open for enabling a certain variety of chroma subsampling. Doesn't seem to be going anywhere though... https://github.com/google/guetzli/issues/79.)

For great writeups on chroma subsampling, here are some links:

http://users.wfu.edu/matthews/misc/jpg_vs_gif/JpgCompTest/JpgChromaSub.html

http://www.red.com/learn/red-101/video-chroma-subsampling

(Similar to above: https://www.cinema5d.com/chroma-subsampling/ )

http://wolfcrow.com/blog/chroma-subsampling-numbers-explained/