Webp images are not being served

[mikelpom]

Hi,

I use Optimus to create webp versions of images that are uploaded to media and with the option to add the .webp extension enabled but then the Cache Enabler plugin does not serve them correctly.

I have tested the regular expression that you use in cache_enabler_disk.class.php with an online tester and it does identify all the images but then when adding .webp it only does it for some and not all. There is a .webp version of all images.

I'm testing with the theme Twenty Twenty-One from WordPress.org and without any plugin that could interfere with content filtering.

[coreykn]

It's possible this is related to the actually delivery of the WebP cached page. Do the WebP cached pages exist in the cache directory (e.g. wp-content/cache/cache-enabler)? These would look like https-index-webp.html (and https-index-webp.html.gz if Gzip pre-compression is enabled.)

[germanoronoz]

Yes, they do exist, and we also can see the comment at the bottom: <!-- Cache Enabler by KeyCDN @ 05.03.2021 15:42:46 (https webp gzip) -->

We are also adding this filter just in case, even though we believe it is not needed: `/**

• Add support for lazy load attributes */ add_filter('cache_enabler_convert_webp_attributes', 'pom_convert_webp_attributes'); function pom_convert_webp_attributes( $attributes ){

  $attributes[] = 'data-bg'; $attributes[] = 'data-src'; $attributes[] = 'data-srcset';

  return $attributes; }`

The only .webp images rendered on the html are the ones that are as inline bg images, such as: <div class="bg-image lazy" data-bg="url(https://DOMAIN.com/IMAGE.jpg.webp)" style="background-image: url(&quot;https://DOMAIN.com/IMAGE.jpg.webp&quot;);" data-was-processed="true"></div>

And this is working thanks to this snippet we added, not by default: `/**

• Replace inline bg images by its webp alternatives */ add_filter('cache_enabler_page_contents_after_webp_conversion', 'pom_change_webp_background_images'); function pom_change_webp_background_images( $content ){

  $regex_rule = '#url([\'"]?(.*?.(?:png|jpg|jpeg))#';

  preg_match_all( $regex_rule, $content, $matches);

  if( empty( $matches ) ) return $content;

  foreach( $matches[1] as $url ){ $content = str_replace( $url, $url . '.webp', $content); }

  $regex_rule_double_webp = '#url([\'"]?(.*?.(?:png|jpg|jpeg).webp.webp)#'; preg_match_all( $regex_rule_double_webp, $content, $matches_double_webp);

  if( !empty( $matches_double_webp ) ) { foreach( $matches_double_webp[1] as $url ){ $newUrl = substr($url, 0 , (strrpos($url, "."))); $content = str_replace( $url, $newUrl, $content); } }

  return $content; }

`

The problem happens with our theme and also with "Twenty Twenty-One" from WordPress.org and without any plugin that could interfere with content filtering.

It also happens on our local environment.

Hope you can fix the issue, Thanks in advance!

[coreykn]

It's not related to the WebP cached variant being served then. Since @mikelpom has ran the regular expression through your page contents and see the matches it sounds like Cache Enabler doesn't believe the WebP images exist. To troubleshoot this further, can you please provide a direct URL to the page that this issue is occurring on?

[mikelpom]

In https://pomstandard.com/es/ for example this image does load the webp version: https://pomstandard.com/files/2019/05/trozo-pepino.png.webp but not this one: https://pomstandard.com/files/2019/06/trozo-pera.png.

If you look for this image in the webp version, it does exist: https://pomstandard.com/files/2019/06/trozo-pera.png.webp

[coreykn]

Hmm, to confirm https://pomstandard.com/es/ doesn't have any filters active and just Cache Enabler is converting inline image URLs to WebP in the default way?

[germanoronoz]

That's it.

But it is also happening on a local environment with the Twenty Twenty-One theme and without any plugin that could interfere with content filtering.

Is it any way to bypass Cache Enabler and make it believe the WebP images exist always?

Thanks

[coreykn]

Thank you. I'm unsure how I missed this previously, but after checking this further it appears this is related to how some images are being invoked, which in this case is through JavaScript:

Cache Enabler is not parsing JavaScript files so it's not able to convert those images URLs to WebP. In this case I'd actually recommend delivering your WebP formatted images by rewriting the MIME type instead. That would allow your server to handle the WebP delivery instead, regardless of how the images are invoked. If you need assistance with getting this setup feel free to contact Optimus support through our contact form and we'd be happy to assist. (That is the support channel for paid licenses for Optimus.)

[germanoronoz]

Thanks, that is definitely an option.

But we are not invoking images through javascript, we are using a lazy loading script.

Take this HTML for example:

<img class=" lazy" title="macbook-tu-imagen" width="800" height="810" src="data:image/svg+xml;utf8,<svg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20800%20810'/>" data-srcset="https://pomstandard.com/files/2019/10/macbook-tu-imagen.png 800w, https://pomstandard.com/files/2019/10/macbook-tu-imagen-296x300.png 296w, https://pomstandard.com/files/2019/10/macbook-tu-imagen-768x778.png 768w, https://pomstandard.com/files/2019/10/macbook-tu-imagen-600x608.png 600w, https://pomstandard.com/files/2019/10/macbook-tu-imagen-100x100.png 100w" data-sizes="(max-width: 800px) 100vw, 800px" data-src="https://pomstandard.com/files/2019/10/macbook-tu-imagen.png" data-description="">

As you can see we are setting the SRC to an inline SVG and the actual image as data-src and data-srcset.

But all this HTML is coming right from PHP, not JS.

And that is the reason we tried this filter as well:

add_filter('cache_enabler_convert_webp_attributes', 'pom_convert_webp_attributes'); function pom_convert_webp_attributes( $attributes ){ $attributes[] = 'data-bg'; $attributes[] = 'data-src'; $attributes[] = 'data-srcset'; return $attributes; }

So I still think something is wrong with cache enabler...

Thanks again and sorry for the inconvenience!

[coreykn]

Using cache_enabler_convert_webp_attributes that way would be unnecessary because by default any data-* attribute will be included.

Does temporarily modifying inc/cache_enabler_disk.class.php#L1006 in the current version (1.6.2) to always be true, like replacing it with if ( 1 ) {, fix this? That is what checks if the image exists when the .webp extension is appended.

If not, would it be possible for you to output the strings passed in the cache_enabler_page_contents_before_store and cache_enabler_page_contents_after_webp_conversion filters, like into text files, and attach them?

[germanoronoz]

Hello,

Thanks for your reply.

Using cache_enabler_convert_webp_attributes that way would be unnecessary because by default any data-* attribute will be included.

I thought so, but we tried just in case ;)

Does temporarily modifying inc/cache_enabler_disk.class.php#L1006 in the current version (1.6.2) to always be true, like replacing it with if ( 1 ) {, fix this? That is what checks if the image exists when the .webp extension is appended.

Yes, that does fix the issue!

Why is is_file function returning false if those files do exist?

Thanks again, Regards

[coreykn]

You're welcome, @germanoronoz. Thanks for that confirmation. Happy to know we've identified why this is occurring.

With the information I have I'm unsure why the is_file() function is returning false. It would be good to see what is actually being checked in this case. Can you please output the $image_path_webp variable to see what is actually being checked. One way to do this would be enabling debugging and printing the variable to the log. For example, the following would be added to your wp-config.php file:

define( 'WP_DEBUG', true );
define( 'WP_DEBUG_LOG', true );

Next, you'd use the error_log() function to send that string to the log, for example:

...

$image_path_webp = self::get_image_path( $image_url_webp );

error_log( '$image_path_webp: ' . $image_path_webp );

// check if WebP image exists
if ( is_file( $image_path_webp ) ) {

...

Lastly, you'd visit a page that is not cached to have the static HTML file generated. The data you've sent to the log will now be available in the wp-content/debug.log file.

[germanoronoz]

Hello,

Here are the results, tested on our main site (https://pomstandard.com/es/):

[29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/url(https://pomstandard.com/files/2019/05/textura-bg.jpg.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/05/pepino-web.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/05/pepino-web-92x300.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/05/pepino-web-313x1024.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/05/pepino-web.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/05/trozo-pepino.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/05/trozo-pepino-150x150.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/05/trozo-pepino-100x100.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/05/trozo-pepino.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/06/trozo-pera.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/06/trozo-pera-150x150.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/06/trozo-pera-100x100.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/06/trozo-pera.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/url(https://pomstandard.com/files/2019/09/lineas-madera.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/09/mano-rock.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/09/mano-rock-98x300.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/09/mano-rock.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/url(https://pomstandard.com/files/2019/05/textura-bg.jpg.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/05/trozo-pepino.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/05/trozo-pepino-150x150.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/05/trozo-pepino-100x100.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/05/trozo-pepino.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2018/06/separador-vertical.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/06/trozo-pera.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/06/trozo-pera-150x150.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/06/trozo-pera-100x100.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/06/trozo-pera.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2018/06/separador-vertical.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/10/imac-gestiona.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/10/imac-gestiona-300x241.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/10/imac-gestiona-768x617.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/10/imac-gestiona-600x482.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/10/imac-gestiona.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/10/macbook-tu-imagen.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/10/macbook-tu-imagen-296x300.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/10/macbook-tu-imagen-768x778.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/10/macbook-tu-imagen-600x608.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/10/macbook-tu-imagen-100x100.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/10/macbook-tu-imagen.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/05/trozo-pepino.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/05/trozo-pepino-150x150.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/05/trozo-pepino-100x100.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/05/trozo-pepino.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/06/trozo-pera.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/06/trozo-pera-150x150.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/06/trozo-pera-100x100.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/06/trozo-pera.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2018/02/pom_standard_payment_methods.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2018/06/logo-footer.png.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2020/03/img_4909.jpg.webp [29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/05/trozo-pepino.png.webp

[coreykn]

Thank you, that is extremely helpful to see. The paths that contain url( would be the reason for those in particular returning false from the is_file() function because the current handling does not take into account attributes like the following:

<div class="bg-image  lazy" data-bg="url(https://pomstandard.com/files/2019/05/textura-bg.jpg)" ...

Instead, it's expecting something like the following:

<div class="bg-image  lazy" data-bg="https://pomstandard.com/files/2019/05/textura-bg.jpg" ...

When creating this handling I didn't consider attributes having that type of value, so I will see what can be done to improve this handling. As for the others I'm unsure why they are also returning false as like you previously said, they do appear to exist, for example:

<img ... data-srcset="https://pomstandard.com/files/2019/06/trozo-pera.png 250w ...

[29-Mar-2021 15:44:20 UTC] $image_path_webp: /home/standardu/domains/pomstandard.com/public_html/files/2019/06/trozo-pera.png.webp

curl -I https://pomstandard.com/files/2019/06/trozo-pera.png
HTTP/2 200
server: nginx
date: Mon, 05 Apr 2021 22:02:28 GMT
content-type: image/png
content-length: 67119
last-modified: Thu, 20 Feb 2020 17:53:46 GMT
etag: "1062f-59f0596270d7b"
accept-ranges: bytes
cache-control: max-age=31104000, public
expires: Tue, 05 Apr 2022 22:02:28 GMT
link: <https://pomstandard.com/files/2019/06/trozo-pera.png>; rel="canonical"

curl -I https://pomstandard.com/files/2019/06/trozo-pera.png.webp
HTTP/2 200
server: nginx
date: Mon, 05 Apr 2021 22:02:50 GMT
content-type: image/webp
content-length: 48626
last-modified: Thu, 20 Feb 2020 17:53:46 GMT
etag: "bdf2-59f0596298e1e"
accept-ranges: bytes
cache-control: max-age=31104000, public
expires: Tue, 05 Apr 2022 22:02:50 GMT
vary: Accept-Encoding,User-Agent
link: <https://pomstandard.com/files/2019/06/trozo-pera.png.webp>; rel="canonical"

I would assume you've also checked your server itself actually has that images stored in the remote path above?

[germanoronoz]

Hello,

Regarding the data-bg attribute, it is not a problem, we can change that to contain only the URL to the file.

Yes, we did check our server and the files do exist:

https://pomstandard.com/files/2019/06/trozo-pera.png.webp

Those files are not physically on /files/2019/... but on /wp-content/uploads/2019/... though, and we have some rewrite rules on our .htaccess:

RewriteRule ^files/(.*) /wp-content/uploads/$1 [L,QSA] RewriteRule ^(.*)/files/(.*) $1/wp-content/uploads/$2 [L,QSA]

And this filter: // Replace upload folder on frontend add_filter('upload_dir', array($this, 'replace_upload_folder'));

Could that do something with the issue? Thanks again!

[coreykn]

Yes, Cache Enabler believing the files do not exist could be related to how the custom directories are being set. I actually had previously thought it could be related to that but did not think much of it as there were other identical paths being converted to WebP.

To see if that's the case, does this issue still occur if you temporarily disable how you've set these custom directories?

[coreykn]

I'm going to close this as I don't think it's related to Cache Enabler. Feel free to open it again if this needs to be investigated further.

[germanoronoz]

OK Corey, sorry I did not respond last time.

We will also look further into this and get back to you if we discover anything.

Regards!

[mmojadad]

Hello I have the exact same issue with Webp images . Theme : Generatepress IIS 10

Regards!

[coreykn]

@mmojadad, can you please provide a direct URL to the page that this issue is occurring on? Furthermore, a screenshot of your Cache Enabler settings would also be helpful.

[mmojadad]

@coreykn Yes, Sure!

https://networkanalysers.com/kb/ptz-pan-tilt-camera/ https://networkanalysers.com/kb/motion-detection-camera/

https://networkanalysers.com/%da%a9%d8%a7%d8%b1%d8%a7-%d8%a7%d9%be-%d8%a7%d9%86%d8%af%d8%b1%d9%88%db%8c%d8%af-%d8%a2%db%8c%d9%81%d9%88%d9%86-%d8%a7%d9%be%d9%84%db%8c%da%a9%db%8c%d8%b4%d9%86-%d9%86%d9%88%d8%aa%db%8c%d9%81%db%8c/

https://networkanalysers.com/%da%af%d9%88%d9%87%db%8c%d9%86%d8%a7%d9%85%d9%87-ssl-%d8%a7%d9%85%d9%86%db%8c%d8%aa%db%8c-%d8%b5%d8%af%d9%88%d8%b1-%d8%b1%d8%a7%db%8c%da%af%d8%a7%d9%86-%d8%a7%d9%87%d9%88%d8%a7%d8%b2-%d9%87%d8%a7/

[coreykn]

@mmojadad, thank you! When checking both https://networkanalysers.com/kb/ptz-pan-tilt-camera/ and https://networkanalysers.com/kb/motion-detection-camera/ I'm unable to replicate any issues. Cache Enabler is rewriting image URLs with the WebP version if available:

Please note it only occurs if the page is actually cached, so you will want to check for the HTML comment at the bottom of your page. It doesn't look like the last two URLs are being cached (likely due to the type of URL), which is why those do not have the WebP images.

[mmojadad]

@coreykn

Yes, Your are right but for the last two URLs caching i taked alook and sceenshots is attached ( For following URL ), Cached generated by Cache enabler, Why do not serve?

https://networkanalysers.com/%da%a9%d8%a7%d8%b1%d8%a7-%d8%a7%d9%be-%d8%a7%d9%86%d8%af%d8%b1%d9%88%db%8c%d8%af-%d8%a2%db%8c%d9%81%d9%88%d9%86-%d8%a7%d9%be%d9%84%db%8c%da%a9%db%8c%d8%b4%d9%86-%d9%86%d9%88%d8%aa%db%8c%d9%81%db%8c/

Regards,

[mmojadad]

Do Cache Enabler Stores file / folder names in Unicode encoding?

[coreykn]

@mmojadad, with the information at hand I can't say for certain why those aren't being delivered. If this occurs in version 1.8.0, which is the next version to be released, can you please open a new issue so I can investigate this further?

[mmojadad]

@coreykn Yes,Sure!