Feature request: Perceptual image encoding.

[BezPowell]

I thought I'd add this here as it's own feature request as my comment in #109 probably isn't too visible.

Feature Description

As a Wordpress user, I would love for it to support perceptual image encoding; either natively, or via a plugin.

Quite a few studies, such as Johannes Siipola's excellent piece Is WebP really better than JPEG? have shown that the quality settings of lossless formats (such as Jpeg and Webp) often result in output images with inconsistent perceptual quality (as measured with SSIM scores) depending upon the source image. For example, photos with heavily blurred backgrounds can often be encoded with a much lower quality setting without any loss in perceptual quality.

As a result of this, finding a “one size fits all” encoding level always involves some compromise; either encoding low detail images at a higher file size than necessary, or reducing perceptual quality in high detail images.

Projects such as jpeg-recompress or pio try to overcome this by encoding each image multiple times, and selecting the smallest output that is over a specified SSIM score. The downside, of course, is that each image has to encoded several times, but the advantage is that you can maintain a fairly high image quality while still keeping filesizes as small as possible. As images are generally encoded once, but sent to clients many times, this is often a worthwhile tradeoff.

Advantages to users

With perceptual image encoding each image is effectively individually optimised. This means a user can select the level of detail-loss they are happy with, and each image is individually encoded to maintain that level of quality; High detail images maintain their detail, and low-detail images are not saved at an unnecessarily large file-size. This is especially useful for sites where image quality is key, but source images vary wildly in their levels of detail, such as photography portfolios.

Neither Jpeg quality levels nor SSIM scores are particularly intuitive, however, SSIM scores should remain the same regardless of output format, whereas quality levels are different for each (e.g. quality 70 will not be the same in Webp as it is in Jpeg).

Advantages to visitors

Image sizes are kept as small as possible within an acceptable quality loss threshold. Wordpress sites stay fast.

Disadvantages

Each image has to be encoded multiple times. This would dramatically increase image encoding times and potentially strain low-powered servers.

[eclarke1]

Tagging @adamsilverstein here for Images. Adam, I've added this to the Images project but please do let me know feedback.

[adamsilverstein]

Hi @BezPowell -

Thanks for opening this issue!

As a Wordpress user, I would love for it to support perceptual image encoding; either natively, or via a plugin.

This is definitely something that could be built as a plugin using the existing hooks WordPress provides, even something existing plugins or image CDNs do: picking the best available format for each request.

As a result of this, finding a “one size fits all” encoding level always involves some compromise; either encoding low detail images at a higher file size than necessary, or reducing perceptual quality in high detail images.

Unfortunately, that is the nature of WordPress - we have chosen a default quality setting (82 for JPEG) and that is applied to all images unless you change the default with a plugin or code. Since WordPress runs across such a wide array of systems, we aim to provide the most compatible approach that will work well for everyone. It may not be the best solution for every case though.

Neither Jpeg quality levels nor SSIM scores are particularly intuitive, however, SSIM scores should remain the same regardless of output format, whereas quality levels are different for each (e.g. quality 70 will not be the same in Webp as it is in Jpeg).

Good point! I do think 70 JPEG / WebP are close to equivalent though.

Each image has to be encoded multiple times. This would dramatically increase image encoding times and potentially strain low-powered servers.

This would make image uploads much slower, already a pain point in WordPress since image processing requires that the user stay on the page during the upload. Probably not a blocker for a plugin, but probably something that would prevent a feature like this from landing in core.

Because this repository is specifically focused on features for the PL plugin destined for core, I am closing this as I believe this feature needs to be implemented in a plugin and isn't suitable for core. Please re-open if I have missed something.