Optimize compression levels for WebP, test file sizes vs. JPEGs

[adamsilverstein]

Trac ticket: https://core.trac.wordpress.org/ticket/54356

[getsource]

I'm doing some exploration here to see if we can adapt https://github.com/cramforce/avif-webp-quality-setting to get the data we need.

[mehigh]

Good read on the topic: https://www.baekdal.com/thoughts/retina-displays-and-file-sizes/

Having a second option for compression for @2x+ retinas might be useful. And thinking from a simplicity's point of view, for example images >3000px in actual pixel width (assuming an implementation that doesn't deliver responsive srcsets,etc.) are probably the ones where the biggest impact can a more aggressive compression apply to in order to really make a difference.

Did a test for a couple of ±1000px images at 75% quality webp yielded 30-35K, while a ±15% quality yielded 15-20K, which isn't a significant enough bump to warrant the complexity of the implementation.

[getsource]

@mehigh Thank you! This is really interesting. Since you mentioned srcset, just in case the info is helpful: WordPress supports srcset natively, and the sizes should be automatically filled out with a set of thumbnails/generated sizes that are created on upload.

Some updates: I got some initial results in this branch with Imagick before a break -- enough to find out that the method should work for us: https://github.com/getsource/avif-webp-quality-setting/tree/generate-with-wordpress

It's a very rough proof of concept at the moment.

I’m currently trying to get it to work with GD as well. Ran into a problem where GD in the WordPress Docker Environment doesn’t seem to support WebP on the default PHP, at least on my installation, so digging into that. As far as I'm aware, this was added back when WebP support was added to WordPress.

If you download that branch, and toss it in any sort of local server, you can view the results from the initial dataset by visiting index.html.

I’m hoping running it on GD, along with expanding the image set, will help verify those results.

On Imagick, initial results show the current default is close, anyway, but that different types of images change the optimal level, so a larger sample set of images would be helpful.

[getsource]

I've been making good progress here, and about ready to move the branch changes into main on my fork: https://github.com/getsource/avif-webp-quality-setting/tree/generate-with-wordpress-gd

There's documentation there now, and it should be possible to use .env to set up to run on environments other than WordPress' Docker development environment.

Here are a couple example reports generated by the script, both with DSSIM 3.1.2: Imagick GD Extended Test Quality Range (quality by 5s), with Imagick Only.

As a next step, I'm:

• Changing the sizes generated to match the default WordPress sizes
• Changing quality testing to 1s (right now, it tests by 10s or 5s, from quality 10-100), so we can test specifically with the 82, the current JPEG quality setting
• Reducing the range (right now, testing if 70-100 can still run) so that the amount of images generated isn't a problem

After that, I think we still should test with some additional images, so picking appropriate ones would be a good idea. I've looked at a few image sample sets, but I'm not sure which is most appropriate at this stage. At the very least, I think we're missing a sample image line art / text.

Any recommendations on sample sets and / or single additional high resolution images would be great.

As a last note, right now this uses DSSIM, but it's pretty straight-forward to swap that out for any other image comparison tool. So if there's something better out there (a little looking found SSIMULACRA, for instance, although that looks to be a couple years old), we can run the script against that as well.

[getsource]

I ported the changes in the branches into a clean branch so that I could make a PR, to bring back to main on my repo: https://github.com/getsource/avif-webp-quality-setting/pull/1

I’m not sure if it’d be better for me just to cherry pick the commits at this point (and what repo it'll eventually live in), but that way it’s a bit more clear what’s going on, if anyone would like to take a look.

I’m pretty excited about this particular revision. I made the changes mentioned above with changing the sizes generated to WP's defaults, and range to [70-100] (incrementing by 1s).

Ended up with these reports: Imagick GD

The last tab (“Quality Settings for JPEG 83”) is the most helpful, I think.

It has all of the sizes that WordPress creates by default, at WP’s default JPEG quality, with a suggested WebP closest quality, and percentage of file-size saved for each of the tested images.

As mentioned in the last update, I think it still needs at least a line art example to be a bit more definitive, but I like where the report ended up.

A note that it takes a long time (hours) to run this particular one, but it’s totally doable to re-run with the images we want / need.

[adamsilverstein]

Ended up with these reports: Imagick GD

The last tab (“Quality Settings for JPEG 83”) is the most helpful, I think.

This is great, thanks for building out this research tool and running initial test sets for JPEG/WebP comparison.

Looking at the two results pages for "Quality settings" JPEG 83, I can draw two clear conclusions:

• WebP quality setting should probably be around the same as JPEG (83), based on the best average quality setting
• WebP saves on average 26-29% on file sizes, with more significant gains on large images than thumbnail size (or medium sized) images. Setting quality by size (issue) could be useful here.

Attaching some screenshots here of those results tabs from the sheets: Imagick:

GD:

When possible it would be great run this on a larger set of test images and in particular test some "outliers" that don't compress as efficiently with WebP.

[markhowellsmead]

When I convert high-quality, pre-optimized JPEGs (Lightroom exports) at 2560px wide, the resultant WebP files are often around 30%-50% larger than the original JPEG files. (I convert using mogrify on the command line at 82% quality.)

[adamsilverstein]

Hey @markhowellsmead - thanks for the note - how do you upload these images to WordPress and place them in posts? Do you use the "Full sized image" after uploading? How are you creating the WebP images?

Can you provide a sample JPEG that you have optimized?

[markhowellsmead]

I export the JPEG from Adobe Lightroom then run ImageMagick's convert (or mogrify) command on it. Using the simplest example, convert 20220328-DSCF1424.jpg 20220328-DSCF1424.webp. Adding the -quality flag makes the file smaller, but - as is the case with JPG - reducing the quality in this way introduces artefacts to the resulting WebP file.

I upload and use the file through the media manager in the usual way and I can use all of the available image sizes, which are all WebP files.

Example image files:

• JPG (755 Kb): https://www.dropbox.com/s/1ady164fwkatlql/20220328-DSCF1424.jpg?dl=0
• WebP (1.2 Mb): https://www.dropbox.com/s/vas8o09hfnh1kna/20220328-DSCF1424.webp?dl=0

[mitogh]

Hey, thanks for providing these examples @markhowellsmead however looking into your WebP example it looks like is actually a PNG image that explains the size of the image.

I used your JPEG image and uploaded it to WordPress these were the results after uploading it to WordPress.

Take into account that the JPEG images are using a lower quality to compress the image 82 and WebP uses 86 which by default creates higher quality compressed images.

[markhowellsmead]

looking into your WebP example it looks like is actually a PNG image that explains the size of the image.

Nope, it's definitely a WebP file. Dropbox is either sending the wrong headers or automatically converting the WebP to a PNG. 🤦‍♂️

Please compare https://dev.permanenttourist.ch/20220328-DSCF1424.jpg and https://dev.permanenttourist.ch/20220328-DSCF1424.webp

Update: these files are no longer online.

Take into account that the JPEG images are using a lower quality to compress the image 82 and WebP uses 86 which by default creates higher quality compressed images.

I'm using quality 82 on the command line to convert the full-sized image to a full-sized WebP file.

[adamsilverstein]

Dropbox is either sending the wrong headers or automatically converting the WebP to a PNG.

Hey @markhowellsmead could you please upload the original image to this GitHub issue directly? you should be able to just drop it onto the comment box! I'd like to test uploading the original to WordPress to see how it compresses.

[markhowellsmead]

Here's the JPG. Github doesn't support WebP.

[adamsilverstein]

Thank you!

I was actually able to reproduce using the JPEG you linked in your last comment. This is the result of WordPress compressing that image locally:

Note that the full size image is larger in WebP, but every other size is smaller

This validates the need for https://github.com/WordPress/performance/issues/186 where we choose the smaller image for every size. Might be especially relevant for the full size image.

[adamsilverstein]

@markhowellsmead thanks so much for providing this sample, this is very helpful.

I'm curious if this is the original image from your camera or if you have reduced it already in size. if you reduced it, what compression quality level did you use? Do you get the same results if you use a much higher level (maybe try using 100 for this intermediate image)? I'm also curious to test what happens if you upload your image at a slightly larger size (eg 2500px) so WordPress creates the -scaled version as the full sized image.

Also, one question about your workflow - after you upload your images, do you typically place the "Full" sized image in your layouts? what do you use for layouts where you might want a smaller image?

[markhowellsmead]

I'm curious if this is the original image from your camera or if you have reduced it already in size. if you reduced it, what compression quality level did you use?

I exported it from Adobe Lightroom, with a target file size. The application automatically chooses the appropriate compression to achieve the requested file size.

Do you get the same results if you use a much higher level (maybe try using 100 for this intermediate image)? I'm also curious to test what happens if you upload your image at a slightly larger size (eg 2500px) so WordPress creates the -scaled version as the full sized image.

I’ll have to provide that information next week. Also, one question about your workflow - after you upload your images, do you typically place the "Full" sized image in your layouts?

Rarely, but sometimes. e.g. in a cover block with full alignment.

what do you use for layouts where you might want a smaller image

Not sure I understand that question. Can you be more specific please?

[mehigh]

Cloudflare does something interesting with its Polish implementation

https://support.cloudflare.com/hc/en-us/articles/4412244347917-Troubleshoot-common-Cf-Polished-statuses

webp_bigger: Polish attempted to convert to WebP, but the image was optimized at the origin server or was created using a low quality setting. Because the WebP version does not exist, the status is set on the JPEG/PNG version of the response.

A similar comparison can be done here... if the original file is optimised to be smaller than a webp, keep delivering the original.

What do you think?

[markhowellsmead]

A similar comparison can be done here... if the original file is optimised to be smaller than a webp, keep delivering the original.

It makes sense to me and would provide the site visitor with the quickest possible result. But PageSpeed Insights will probably still moan. Imagify’s solution (which delivers an appropriate file on requesting a JPG, according to browser support) would be especially luxurious.

[mehigh]

I don't think the 'moaning' of a lab tool should be what would drive the evolution, but the user experience. That will be 'ever-green' regardless of how measuring tools change over time. If we're able to deliver a faster user experience and not decline the performance of already optimised setups, it's a win-win in my book. I'm curious if anyone would be able to complain about this being the new normal.

[mitogh]

A similar comparison can be done here... if the original file is optimized to be smaller than a WebP, keep delivering the original.

This plugin already contains a logic to perform and deliver the smallest file available in case the JPEG one is smaller than the WebP version.

• 186

[adamsilverstein]

Imagify’s solution (which delivers an appropriate file on requesting a JPG, according to browser support) would be especially luxurious.

I don't think we can do this type of request based response directly because WordPress doesn't handle image requests - those are handled by the we web server. A plugin might support some use cases (eg. Apache) but it would be hard/impossible for core to implement given the variety of stacks that run WordPress.

It makes sense to me and would provide the site visitor with the quickest possible result.

Yea that is the main goal of this work in the first place, so if you place a full size image on your site and the jpeg is smaller than the webp we should serve the jpeg for that size. Pretty sure that is what the plugin does currently, you can test by uploading your example and placing the full sized image. I'll also give this a test.

Rarely, but sometimes. e.g. in a cover block with full alignment. what do you use for layouts where you might want a smaller image Not sure I understand that question. Can you be more specific please?

I'm asking here about your use of images at sizes other than "Full" size.

If you use the large or medium size or any other custom size on your site, the generated WebP will still be smaller than the generated JPEG in my testing so far.

Which images actually get served to your visitors will depend on many factors including their viewport size (with a srcset) . The Full size image would mainly be used when placed that way intentionally.

[nosilver4u]

Per our discussion on Slack, here is our data on non-WebP User Agents shared from Logz%2Cquery%3A(match_phrase%3A(appname%3Aresizer)))%2C('%24state'%3A(store%3AappState)%2Cmeta%3A(alias%3A!n%2Cdisabled%3A!f%2Cindex%3A'logzioCustomerIndex'%2Ckey%3Awebp_ua%2Cnegate%3A!f%2Cparams%3A(query%3A'0')%2Ctype%3Aphrase)%2Cquery%3A(match_phrase%3A(webp_ua%3A'0')))%2C('%24state'%3A(store%3AappState)%2Cmeta%3A(alias%3A!n%2Cdisabled%3A!f%2Cindex%3A'logzioCustomerIndex'%2Ckey%3Aext%2Cnegate%3A!f%2Cparams%3A(query%3Ajpg)%2Ctype%3Aphrase)%2Cquery%3A(match_phrase%3A(ext%3Ajpg))))%2Clinked%3A!f%2Cquery%3A(language%3Alucene%2Cquery%3A'')%2CuiState%3A()%2Cvis%3A(aggs%3A!((enabled%3A!t%2Cid%3A'1'%2Cparams%3A()%2Cschema%3Ametric%2Ctype%3Acount)%2C(enabled%3A!t%2Cid%3A'2'%2Cparams%3A(field%3Auseragent%2CmissingBucket%3A!f%2CmissingBucketLabel%3AMissing%2Corder%3Adesc%2CorderBy%3A'1'%2CotherBucket%3A!f%2CotherBucketLabel%3AOther%2Csize%3A20)%2Cschema%3Asegment%2Ctype%3Aterms))%2Cparams%3A(addLegend%3A!t%2CaddTimeMarker%3A!f%2CaddTooltip%3A!t%2CcategoryAxes%3A!((id%3ACategoryAxis-1%2Clabels%3A(filter%3A!f%2Crotate%3A0%2Cshow%3A!t%2Ctruncate%3A200)%2Cposition%3Aleft%2Cscale%3A(type%3Alinear)%2Cshow%3A!t%2Cstyle%3A()%2Ctitle%3A()%2Ctype%3Acategory))%2Cdimensions%3A(x%3A(accessor%3A0%2CaggType%3Aterms%2Cformat%3A(id%3Aterms%2Cparams%3A(id%3Astring%2CmissingBucketLabel%3AMissing%2CotherBucketLabel%3AOther%2CparsedUrl%3A(basePath%3A%2Fkibana-7-6%2Corigin%3A'https%3A%2F%2Fapp.logz.io'%2Cpathname%3A%2Fkibana-7-6%2Fapp%2Fkibana)))%2Clabel%3A'useragent%3A%20Descending'%2Cparams%3A())%2Cy%3A!((accessor%3A1%2CaggType%3Acount%2Cformat%3A(id%3Anumber)%2Clabel%3ACount%2Cparams%3A())))%2Cgrid%3A(categoryLines%3A!f)%2Clabels%3A(show%3A!t)%2ClegendPosition%3Atop%2CseriesParams%3A!((data%3A(id%3A'1'%2Clabel%3ACount)%2CdrawLinesBetweenPoints%3A!t%2ClineWidth%3A2%2Cmode%3Anormal%2Cshow%3A!t%2CshowCircles%3A!t%2Ctype%3Ahistogram%2CvalueAxis%3AValueAxis-1))%2CthresholdLine%3A(color%3A%23E7664C%2Cshow%3A!f%2Cstyle%3Afull%2Cvalue%3A10%2Cwidth%3A1)%2Ctimes%3A!()%2Ctype%3Ahistogram%2CvalueAxes%3A!((id%3AValueAxis-1%2Clabels%3A(filter%3A!t%2Crotate%3A75%2Cshow%3A!t%2Ctruncate%3A100)%2Cname%3ALeftAxis-1%2Cposition%3Abottom%2Cscale%3A(mode%3Anormal%2Ctype%3Alinear)%2Cshow%3A!t%2Cstyle%3A()%2Ctitle%3A(text%3ACount)%2Ctype%3Avalue)))%2Ctitle%3A''%2Ctype%3Ahorizontal_bar))&_g=(filters%3A!()%2CrefreshInterval%3A(pause%3A!t%2Cvalue%3A0)%2Ctime%3A(from%3Anow-7d%2Cto%3Anow))&indexPattern=logzioCustomerIndex*&type=horizontal_bar). These User-Agents represent roughly 40% of our traffic, filtered down to JPG-only requests.

[adamsilverstein]

@nosilver4u thank you for sharing this data. Question: so these agents all made requests without image/webp in the Accept request header, right? Oddly I see Google Imagebot as the top offender, yet I know that bot does digest WebP images (as stated here) so I wonder if the issue is with the request headers

[nosilver4u]

@adamsilverstein Correct, it's any agent that doesn't list image/webp in the Accept request header. I thought that odd myself that they were in the "no-WebP" camp, and that article clearly states they support WebP. Perhaps they don't send the header so that they can get the original JPG/PNG if available, rather than a WebP-derivative? In other words, perhaps they "support" ingesting WebP, but don't advertise it?

Update: confirmed that the Google Image bot prefers to get the original JPG/PNG (if available), but will take a WebP if that's what you give it.

[erikyo]

Here is a js script I made to compare compressed formats, it's included into a wp template that you can find here and, with an sh script, can generate all the resizes automatically check the ssim, psnr and mae.

The interesting thing is the possibility to compare dissimilarities with another source, in this case a jpeg @ 80% compressed in all other formats and quality against the original png, let me know what you think!

To me the result somewhat explains why some users are disappointed, if the image was already compressed to the limit of acceptability, any further compression would worsen it. To obtain good results you would have to increase the quality to at least 90% but this requires an extra filesize and it is not always worth it (this applies to all compressed formats).

[getsource]

It was requested to move the conversation over here from Slack, so that the WebP team can take a look, so I'll try to summarize.

While I was working on adapting the image testing script to work with WordPress, I noticed that one of the example sizes didn't seem to be equivalent to the JPEG, even though it was marked to be equivalent quality by the script.

I believe the reason is this line (I added comments in the modified code, below, for more context): https://github.com/cramforce/avif-webp-quality-setting/blob/main/scripts/image-quality-compare.js#L22

If I remove the "margin" to the DSSIM (like this), I get results that are much less favorable to WebP for most of the example images.

With DSSIM Margin: https://docs.google.com/spreadsheets/d/1n-ead6aea2MTIk9Dpcs0HrtfU6Z5nMSZrzgIZ220Qhk/edit#gid=1107534790

Without DSSIM Margin: https://docs.google.com/spreadsheets/d/1bWaKfh3Sq0M0fQ8yqr7fEb0kmmVsvR_3J2qNgKu6PBQ/edit#gid=1107534790

In the second example run, it's only GD, and it has two additional images, so the sums of savings are not directly comparable.

sample4 is the WordPress logo pasted on top of of sample0 a few times, and sample5 is the WordPress logo by itself.

I asked the original author if they remembered why this choice was made, but unfortunately they did not (which is understandable, and happens).

Assuming I am both understanding the code properly, and not breaking the data/statistics by making this change, my question is this: Why would we want to have a margin in DSSIM, when comparing to see which image quality is equivalent? It seems to me that the primary interest is making sure the quality is equal, or just as good.

The only reason I can think of is perhaps if the difference isn't perceptible, and it makes a big difference to move one or two quality % points for very little savings. It's not clear to me if there is consensus on what a perceptible difference in DSSIM is (in particular, for the current version of DSSIM).

I can do some more digging with spot checking around the suggested "best quality" levels, to see if it becomes more clear, but because I've been stuck for a bit, I thought I'd see if anyone had any suggestions.

[erikyo]

@getsource i guess the reason of the "margin" is to get in addition to all the images with a good DSSIM result (info.dssim <= ref.dssim) also all the images within a range (0.000243 was the value established by the author as reliable for "similar" images).
This "margin" probably makes some webps move from discarded to passed and represents an "acceptable loss of quality" (to answer your question, this is indeed true).

And if you want to contibute i've prepared a template for WordPress that can show the result of image comparison into a website, the images and results are generated with an sh script (with imagemagick, the same way that wordpress use). Analysis are performed with the imagemagick compare function but since it's a custom sh script can be extended (adding for example Guetzli or other image quality checks)

[erikyo]

I export the JPEG from Adobe Lightroom then run ImageMagick's convert (or mogrify) command on it. Using the simplest example, convert 20220328-DSCF1424.jpg 20220328-DSCF1424.webp. Adding the -quality flag makes the file smaller, but - as is the case with JPG - reducing the quality in this way introduces artefacts to the resulting WebP file.

@markhowellsmead this is because the jpg image quality is about 55% (this image @100% bottom center) and to get the real size comparison you would need to save the image at the same a quality setting ( but to get the real comparison in image similarity instead you should save the original raw directly to webp)

The difference in quality is due to the fact that you use as original image a "compromised" jpg with already a lot of artifacts and, in addition to it, you add the webps artifacts. This is something expected with overcompressed jpg images as source and the reason why I recently realised that there is no point in compressing jpg images that have been compressed below a certain threshold because the corresponding jpg is better in quality and weight.

Imagemagick seems to have a quality of 92% for webp if it is not set (try the command identify -format '%Q' 20220328-DSCF1424.webp) that the reason for the larger file when the quality isn't set

[markhowellsmead]

Thanks for the feedback @erikyo. Exporting an optimised JPG will certainly add artefacts, which will then also be present in the resultant WebP file. (An aside: your screenshot isn't the export panel from Lightroom and bears no correlation to the settings or results of my export process.)

The goal is to convert this JPG to WebP and that the WebP file should be smaller than the JPG.

Imagemagick seems to have a quality of 92% for webp if it is not set

I'd overlooked this; thanks. Converting the 755kb JPG without specifying a quality results in a 1.4mb WebP file at 92% quality. (So a secondary degradation when compared to the JPG.) Setting -quality 100 results in a 4.8 Mb WebP file. This is an even worse result.

As we've already discussed here, if the site owner were to choose the “full” size when placing the image on the website, then the use of the WebP would cause a massive performance degradation. This is where the WordPress solution has to be intelligent enough to recognise that the JPG file is smaller and, therefore, automatically choose this image instead of the WebP.

[erikyo]

An aside: your screenshot isn't the export panel from Lightroom

yes but afaik if I can save this file without any quality loss at 55% means the source was saved at that quality. Probably you have checked the flag "limit filesize to [ ]kb" and this has unpredictable results if you need to know the quality of the image.

The goal is to convert this JPG to WebP and that the WebP file should be smaller than the JPG.

I hope the goal is to convert the jpg into webp if is worth to do that. Sometimes (as in this case) the resulting webp will be always worst than the jpg (sadly). to get a good result you need a good source image, the best result are with lossless images as source.

Setting -quality 100 results in a 4.8 Mb WebP file. This is an even worse result.

This will save the image into a lossless format (4:4:4 like png, tiff etc) and that is the reason for that large filesize (isn't a web optimized format but useful to save source images for example)

This is where the WordPress solution has to be intelligent enough to recognise that the JPG file is smaller and, therefore, automatically choose this image instead of the WebP.

I also think that would be nice, and also nice to check the image quality before encode an image into webp to avoid the generation of useless image if is possible, Imagemagick has getImageCompressionQuality() but I haven't found nothing similar in gd. Anyhow the quality is a calculation since isn't saved into image metadata so we can't rely 100% on that function. Note that this applies only to the main image and not to resized images because downscale will add "definition" to images.

[adamsilverstein]

If I remove the "margin" to the DSSIM (like this), I get results that are much less favorable to WebP for most of the example images.

I'm going to take the test images and manually run them through processing with core imagick and gd manually with a couple of different quality settings to see how the resulting sizes compare to the automated runs.

[adamsilverstein]

If I remove the "margin" to the DSSIM (like this), I get results that are much less favorable to WebP for most of the example images.

this is comparing the jpeg compressed image to the webp compressed image, right? In this case I would always expect some dssim difference when comparing them. I wonder we should try comparing a lossless resize of the original to both jpeg and webp compressed versions and look at the dssim vs the original.

[adamsilverstein]

manual testing results: WebP 82 quality (same as jpeg) - https://gist.github.com/adamsilverstein/39b5dfb30cb825423f74f85acf520581 WebP 90 quality - https://gist.github.com/adamsilverstein/4a21ad54f2fbb98dff05876c6376da67

[erikyo]

ciao @adamsilverstein, It's also important to consider the encoded starting jpg quality since this significantly influences the final result. A jpg image above 87-92% quality normally gives good results while below this quality to have an acceptable SSIM (relative to the original lossless image) the size of the webp would have to exceed that of the jpg nullifying what is the advantage of webps. In short we can say that it all depends on the original images the user uploads to WordPress (and there are different kind of user maybe that's why some people complain and others don't). If the images are pre-optimized the result will be poor while if the image is of good quality there will be an advantage with the webp encoding.

In addition, I want to point out that you are right and when we evaluate the SSIM it would be better to compare it with the lossless image (even though the webp is actually generated from the jpg). This way we can assess the generation loss that occurs, whereas otherwise we would not have a real measure of how much was lost in the "double" encoding process, because we need to sum the loss of the jpg encoding and the loss of the webp encoding.

[hugh-gibson]

I took part in the contributors day at WCEU2022 on the core tables and was introduced to the work to move this webp work into core. I come from an ecommerce background with another product, and we have implemented webp for a number of our clients. I installed the performance plugin and tried out a few image uploads, hooking filter_webp_quality to set different quality levels. As pointed out here the default quality level regularly gives bigger webp images compared to the original jpeg.

However, our experience is that some clients want to have a high quality level, and others are happier with a lower level. Therefore setting a generic quality level is restrictive, and even setting it via filter makes it difficult for the client to tweak. In addition, the client needs to be able to see their optimised images and compare them with the original, to get a better idea of what is being lost. We've implemented a preview tool that shows the images in the same location in a web page, using arrow keys to cycle through or switch between the original and a compressed image. It's easy to see any differences by switching back and forward.

If you set a single fixed quality level with little size advantage, then the feature is basically useless. With our work, we can typically get webp that are more than good enough at under 20% of the original jpeg size. That's worth doing.

We're just getting involved with Wordpress so we'll be contributing more. I offered to share our compression comparison tool with the table lead as switching in place is much better than trying to compare images side by side.

[JosKlever]

I was doing some tests to check how the Performance Lab plugin is working and uploaded some images. Most webp versions were a little smaller (about 80% of the original jpg), but with this photo (original from camera/phone) the webp versions were larger, except for the "-scaled" version.

These sizes were created:

 327156 2022-05-06-13.00.54-1024x1024.jpg
 342726 2022-05-06-13.00.54-1024x1024.webp
  57611 2022-05-06-13.00.54-150x150.jpg
  58230 2022-05-06-13.00.54-150x150.webp
 386996 2022-05-06-13.00.54-1536x864.jpg
 401492 2022-05-06-13.00.54-1536x864.webp
 401641 2022-05-06-13.00.54-1568x882.jpg
 414168 2022-05-06-13.00.54-1568x882.webp
 612138 2022-05-06-13.00.54-2048x1152.jpg
 619004 2022-05-06-13.00.54-2048x1152.webp
  76120 2022-05-06-13.00.54-300x300.jpg
  79110 2022-05-06-13.00.54-300x300.webp
 116571 2022-05-06-13.00.54-600x400.jpg
 124470 2022-05-06-13.00.54-600x400.webp
 151359 2022-05-06-13.00.54-600x600.jpg
 164224 2022-05-06-13.00.54-600x600.webp
 212709 2022-05-06-13.00.54-768x768.jpg
 228950 2022-05-06-13.00.54-768x768.webp
3409603 2022-05-06-13.00.54.jpg
 871571 2022-05-06-13.00.54-scaled.jpg
 844670 2022-05-06-13.00.54-scaled.webp

Some plugins are active but none related to image upload/optimization. The server is using the imagick.

[erikyo]

but with this photo (original from camera/phone) the webp versions were larger, except for the "-scaled" version.

Hi @JosKlever, interesting case...

I want to add some additional info about your image for further considerations:

identify  2022-05-06-13.00.54.jpg
2022-05-06-13.00.54.jpg JPEG 4032x2268 4032x2268+0+0 8-bit sRGB 3.25165MiB 0.000u 0:00.000
# returns the jpg image quality 
identify -format '%Q'  img.jpg
92%

Just tested locally with convert imagename.jpg imagename.webp and the webp generated it's about 2488kb. Anyway using the WordPress resize function this is the result:

stat -c "%n,%s" * | column -t -s,
2022-05-06-13.00.54-100x100.jpg     54315
2022-05-06-13.00.54-100x100.webp    54332
2022-05-06-13.00.54-1024x576.jpg    207498
2022-05-06-13.00.54-1024x576.webp   220734
2022-05-06-13.00.54-150x150.jpg     57611
2022-05-06-13.00.54-150x150.webp    58232
2022-05-06-13.00.54-1536x864.jpg    386996
2022-05-06-13.00.54-1536x864.webp   400934
2022-05-06-13.00.54-1920x1080.jpg   552865
2022-05-06-13.00.54-1920x1080.webp  563588
2022-05-06-13.00.54-2048x1152.jpg   612138
2022-05-06-13.00.54-2048x1152.webp  618000
2022-05-06-13.00.54-400x225.jpg     75579
2022-05-06-13.00.54-400x225.webp    78328
2022-05-06-13.00.54-400x400.jpg     95418
2022-05-06-13.00.54-400x400.webp    101728
2022-05-06-13.00.54-475x267.jpg     85589
2022-05-06-13.00.54-475x267.webp    89046
2022-05-06-13.00.54-600x338.jpg     105891
2022-05-06-13.00.54-600x338.webp    111742
2022-05-06-13.00.54-768x432.jpg     140452
2022-05-06-13.00.54-768x432.webp    150776
2022-05-06-13.00.54.jpg             3409603
2022-05-06-13.00.54-scaled.jpg      871571
2022-05-06-13.00.54-scaled.webp     842840

What we have to think about is that we are comparing an 82% jpg (WordPress has this standard for jpg resizes quality) with the webp resize, and both are generated from the original which is 92%.

For this we should carefully evaluate the SSIM of the two results... the quality paramer is just a way of passing settings to the encoder but what we should always take as a reference is the image similarity!

So we could do something like this: ("2022-05-06-13.00.54" was renamed to "original" for better understanding)

$: convert -quality 87 original.jpg img.webp
$: convert -quality 82 original.jpg img.jpg
$: compare -metric SSIM -verbose original.jpg img.webp res.png
      original.jpg JPEG 4032x2268 4032x2268+0+0 8-bit sRGB 3.25165MiB 0.120u 0:00.118
      img.webp WEBP 4032x2268 4032x2268+0+0 8-bit sRGB 1.73574MiB 0.230u 0:00.241
      Image: original.jpg
        Channel distortion: SSIM
          red: 0.97622
          green: 0.978625
          blue: 0.971797
          all: 0.975547

$: compare -metric SSIM -verbose original.jpg img.jpg res.png
      original.jpg JPEG 4032x2268 4032x2268+0+0 8-bit sRGB 3.25165MiB 0.110u 0:00.117
      img.jpg JPEG 4032x2268 4032x2268+0+0 8-bit sRGB 1.80553MiB 0.100u 0:00.096
      Image: original.jpg
        Channel distortion: SSIM
          red: 0.974551
          green: 0.977116
          blue: 0.969274
          all: 0.973647

Seems the webp has a better quality (+0.02 SSIM) with a lower filesize (-0.06 mb) maybe due the different compression method
(https://developers.google.com/speed/webp/docs/compression), and is a clear winner in this case (SSIM/Filesize as parameters). In addition the jpg compression algo is really challenged by the noise (sand rocks etc) and this isn't the best performer with this kind of subjects...
In summary, we can say that every image format has it point of strengths and it's disadvantages but generally webp wins over standard jpg 99.9% of the times.

other examples: this is an example of an image that is jpg "uncompressable" this png has a lower filesize of any (decent) jpg image you save from that!

[adamsilverstein]

@getsource I was able to track down some additional information about the dssim margin question and why it might need to have a small margin versus the jpeg:

• The SSIM algorithm has some specific biases that means it is not reliable in all cases (or may favor specific compression approaches)
• Thorough analysis would weight several measurement approaches including PSNR, Butteraugli, CIDE2000 etc. I'm going to see if I can run some of that analysis or get help running it.
• A jpeg encoder may be able to recompress existing jpeg block distortion due to the block size. it might be worth including tests with sources that are PNGs or WebPs for completeness. Although my guess is jpeg is the top uploaded format.

Why have a margin? A small gap may not be that distinguishable, so having a strict cut off might be overkill at these quality targets.

[erikyo]

A jpeg encoder may be able to recompress existing jpeg block distortion due to the block size. it might be worth including tests with sources that are PNGs or WebPs for completeness. Although my guess is jpeg is the top uploaded format.

if it possible it would be cool to check also different jpg qualities (50% - 75% - 82% - 90%)... the result will be quite different especially when compared with the original lossless (i.e. the lossless from which the jpg was generated). It is true that jpg is the most uploaded format but what we are interested to is to not loss quality and this loss starts during the first conversion (outside WordPress) so if we decided to double encode images we need to take in account this sum of losses.

[getsource]

this is comparing the jpeg compressed image to the webp compressed image, right? In this case I would always expect some dssim difference when comparing them. I wonder we should try comparing a lossless resize of the original to both jpeg and webp compressed versions and look at the dssim vs the original.

The suggestion you made is how the script is currently intended to work. I agree that a DSSIM between the two compressed images would not be as helpful for this particular purpose.

I am double-checking to make sure things are working as I understand / is intended as part of:

manual testing results: WebP 82 quality (same as jpeg) - https://gist.github.com/adamsilverstein/39b5dfb30cb825423f74f85acf520581 WebP 90 quality - https://gist.github.com/adamsilverstein/4a21ad54f2fbb98dff05876c6376da67

This is interesting, thank you! I'll do some followup tests here, since at a first glance it doesn't seem to match the generated results for those images / sizes.

[getsource]

@adamsilverstein Sorry for the wait on this.

I decided to check with uploading the same image (without using the script) to my trunk / docker install with the performance plugin + WebP enabled.

The JPEG sizes are similar to the ones in the directory listings you pasted, but the WebP sizes are larger than the ones in the gists you linked, whether I set things to 82 or 90 WebP quality.

I'm wondering if it might be due to a difference in configuration of our environments. Would you mind posting the configuration from your test site's Site Health?

Here's the info from mine -- it says that the default-to-gd plugin is active, but I was just using it to change the filter for quality for WebP for these tests. The images were generated using Imagick.

82 Quality: https://gist.github.com/getsource/9c0fd74123c40c12357116011da98e68 90 Quality: https://gist.github.com/getsource/44f80fbaae3efbf0207964152e6e25b5 WordPress site info: https://gist.github.com/getsource/58162d046caafe9fbf7e8f1b817eefe9

[adamsilverstein]

I'm wondering if it might be due to a difference in configuration of our environments. Would you mind posting the configuration from your test site's Site Health?

My local environment is highly configurable and I regularly change which PHP version and image library I'm testing; unfortunately I didn't record my settings at the time of those tests so I'm not really sure what I had enabled, sorry about that

How do the 82 quality images compare to the jpeg versions in your visual estimation, do you notice any difference in the quality?

[adamsilverstein]

@getsource Based on everything so far, I would like to propose using the same default for WebP that we use for JPEG - namely 82. Please review my comment here - https://core.trac.wordpress.org/ticket/54356#comment:9 and the suggested code change. Let me know what you think!

[getsource]

Hi @adamsilverstein ! Sorry for the wait here. I'm working on testing this, but having a hard time getting trunk to respect the filter. It's possible it's a local environment issue, but I'm not sure yet.

In the meantime, I thought it'd be helpful to provide my current thoughts. TL;DR: Based on testing so far: I'm not strongly opposed to 82, but I prefer a value a little higher: 84.

The mean / median in the testing were very close to 82. However, they included an an image that was an outlier, and very easy to compress (72 quality required). If this is removed, the mean/median become 84.

The image in the tests (a portrait) that required the highest value to match its equivalent 82 JPEG needed a WebP quality of 90.

For this image, and similar ones, the quality of the resizes will be lower than it was previously.

I like the idea of matching the intent behind WebP's design, but I think we should make sure that aligns with the quality users currently expect from JPEG resizes (Side-wondering: Do you know if that design was based off of resizing from lossless images rather than JPEGs?).

I also don't know where 86 came from, but it seems remarkably close to the value required for JPEG resizes into WebP!

All of that said, I'm going continue another visual test, like you asked previously, and I'll follow up again. I'm going to verify that the editors we're using actually generate a different size based on 82 / 84, and see if those would be visually distinguishable, regardless of the better DSSIM.

[getsource]

Ah, another side-note -- I noticed some of the spreadsheets had "83" instead of "82" in their pivot tables for JPEG quality, so I updated a couple of the more recent ones, so that the data will be more useful for this conversation.

[adamsilverstein]

I'm working on testing this, but having a hard time getting trunk to respect the filter. It's possible it's a local environment issue, but I'm not sure yet.

I was having issues with this as well recently and would up editing core to be "sure" since it is a little hard to tell what level was used.

TL;DR: Based on testing so far: I'm not strongly opposed to 82, but I prefer a value a little higher: 84.

I'm not opposed to 84, my guess is this would provide slightly better quality images overall vs. jpeg and still remain smaller. at all sizes. I still prefer 82 because it simplifies the default quality logic and I believe result in very similar quality images (that are smaller). I do agree that a slightly higher number like 84 would probably better match the "quality profile" of jpeg at 82, but I'm saying the difference won't be perceivable, especially when comparing against the original image.

The image in the tests (a portrait) that required the highest value to match its equivalent 82 JPEG needed a WebP quality of 90.

The tests compare structural similarity, and when an image is highly detailed WebP has to work harder to match JPEG. Other testing tools such as visually comparing with real users or PSNR (signal to noise ratio) tests might give you different results. I guess my question here is, if you used 82 for WebP on this image, would the difference between webp & jpeg be perceivable?

I also don't know where 86 came from, but it seems remarkably close to the value required for JPEG resizes into WebP!

I believe it came from here: https://www.industrialempathy.com/posts/avif-webp-quality-settings/#quality-settings-for-a-range-of-jpeg-qualities

in this chart Malte suggests a WebP quality of 82 to match a JPEG quality of 80 and I extrapolated from there.

All of that said, I'm going continue another visual test, like you asked previously, and I'll follow up again. I'm going to verify that the editors we're using actually generate a different size based on 82 / 84, and see if those would be visually distinguishable, regardless of the better DSSIM.

perfect, thanks for your continued attention here!

[adamsilverstein]

I'm working on testing this, but having a hard time getting trunk to respect the filter. It's possible it's a local environment issue, but I'm not sure yet.

@getsource - I also experienced this and think it might be a bug - I found the comment where I encountered this issue (and fixed it) - https://github.com/WordPress/wordpress-develop/pull/2393#issuecomment-1179310739 that might be helpful

[markhowellsmead]

Rarely, but sometimes. e.g. in a cover block with full alignment. what do you use for layouts where you might want a smaller image Not sure I understand that question. Can you be more specific please? I'm asking here about your use of images at sizes other than "Full" size.

If you use the large or medium size or any other custom size on your site, the generated WebP will still be smaller than the generated JPEG in my testing so far.

That is the ideal situation and clients are taught to choose the appropriate size when adding content to the site. If the full-sized image is set then the performance will be worse, which is expected. If the new feature will choose between the two alternatives (WebP and JPG) for the selected size when the HTML is rendered, I agree that this will be the best solution for performance, if not for measurement tools. (Which should always take second place.)D

[adamsilverstein]

Thanks for all the lively discussion here! We already resolved this issue in the plugin by setting the quality to 82 for WebP, see https://github.com/WordPress/performance/pull/571.

Closing this issue as resolved, using 82 here gives us a good way to test/validate making a similar change in core.