Open mch0lic opened 3 years ago
Hi
Try https://domain.com/?PageSpeedFilters=+debug
. and see in the html code inserted debug messages as comments.
At a first glance I see this:
pagespeed Domain img.domain.com;
pagespeed MapProxyDomain https://domain.com/img/ https://img.domain.com/;
Is img.domain.com running pagespeed? If no, then you don´t need the Domain directive.
Why?
If in your html code there is a https://img.domain.com/some-image.jpg
, with this directive these url is rewrited to some like
https://img.domain.com/xsome-image.jpg.SOMEHAS.jpg
(or webp), then this request come to img.domain.com and get a 404 because this domain is not running pagespeed.
If in your html are not present any url from https://img.domain.com
then this directive does nothing.
For the MapProxyDomain: If the https://img.domain.com/
is really an https server you need to enable https fecht is pagespeed becaause if not enabled pagespeed by default fecht with http only.
https://www.modpagespeed.com/doc/https_support#https_fetch
with pagespeed FetchHttps enable;
you need to enable other directive pagespeed SslCertDirectory directory;
as you say is Ubuntu directory sould be /etc/ssl/certs
And I can´t see a pagespeed Domain domain.com
; and the docs say "In addition to optimizing HTML resources, PageSpeed restricts itself to optimizing resources (JavaScript, CSS, images) that are served from domains, with optional paths, that must be explicitly listed in the configuration file. "
https://www.modpagespeed.com/doc/domains#auth_domains
Try https://domain.com/?PageSpeedFilters=+debug. and see in the html code inserted debug messages as comments.
<img width="284"
height="284"
itemprop="image"
src="https://www.domain.com/img/2c1eb6db-316a-4959-9767-9d82613949f4/284x284/xtastingbox1-openjpg.jpg.pagespeed.ic.iVO7SrJwHL.webp"
class="attachment-shop_single size-shop_single image-without-video"
alt=""
data-src="https://www.domain.com/img/2c1eb6db-316a-4959-9767-9d82613949f4/284x284/xtastingbox1-openjpg.jpg.pagespeed.ic.iVO7SrJwHL.webp"
data-large_image="https://img.domain.com/2c1eb6db-316a-4959-9767-9d82613949f4/284x284/tastingbox1-openjpg.jpg"
data-color="default"
data-large_image_width="925"
data-large_image_height="925"
data-pagespeed-url-hash="339213084"
onload="pagespeed.CriticalImages.checkImageForCriticality(this);">
<!--Image https://img.domain.com/2c1eb6db-316a-4959-9767-9d82613949f4/284x284/tastingbox1-openjpg.jpg does not appear to need resizing.-->
<!--The image was not inlined because you have chosen to only inline the critical images but this image is not critical.-->
<!--Image https://img.domain.com/2c1eb6db-316a-4959-9767-9d82613949f4/284x284/tastingbox1-openjpg.jpg does not appear to need resizing.-->
<!--The image was not inlined because you have chosen to only inline the critical images but this image is not critical.-->
Even though urls are rewritten and it seems pagespeed is attempting to serve webp I still get 404 though:
Is img.domain.com running pagespeed? If no, then you don´t need the Domain directive/
img.domain.com is pointing to another server, it does not run pagespeed indeed.
If in your html code there is a https://img.domain.com/some-image.jpg, with this directive these url is rewrited to some like https://img.domain.com/xsome-image.jpg.SOMEHAS.jpg (or webp)
This is not correct, the https://img.domain.com/some-image.jpg will be rewritten into https://**www**.domain.com/**img**/xsome-image.jpg.SOMEHASH.jpg and if I understand correctly it's cached on the server where pagespeed is enabled (www.domain.com).
with pagespeed FetchHttps enable; you need to enable other directive pagespeed SslCertDirectory directory; as you say is Ubuntu directory sould be /etc/ssl/certs
Please see the updated config at the end of the comment.
And I can´t see a pagespeed Domain domain.com; and the docs say "In addition to optimizing HTML resources, PageSpeed restricts itself to optimizing resources (JavaScript, CSS, images) that are served from domains, with optional paths, that must be explicitly listed in the configuration file. "
I've tried both www.domain.com and domain.com, but it doesn't seem to work.
<script defer="" type="text/javascript" src="https://www.domain.com/app/plugins/woocommerce/assets/js/jquery-blockui/jquery.blockUI.min.js?ver=2.7.0-wc.5.6.0" id="jquery-blockui-js"></script>
<!--The preceding resource was not rewritten because its domain (www.domain.com) is not authorized-->
pagespeed on;
pagespeed XHeaderValue "ngx_pagespeed";
pagespeed RewriteLevel PassThrough;
pagespeed FileCachePath /dev/shm/ngx_pagespeed_cache;
pagespeed FileCacheSizeKb 307200;
pagespeed Statistics on;
pagespeed StatisticsLogging on;
pagespeed EnableCachePurge on;
pagespeed LogDir /var/log/pagespeed;
pagespeed AdminPath /pagespeed_admin;
pagespeed FetchHttps enable;
pagespeed SslCertDirectory /etc/ssl/certs;
pagespeed LoadFromFile "https://www.domain.com/app/uploads/" "/var/www/www.domain.com/shared/uploads/";
pagespeed LoadFromFile "https://www.domain.com/app/plugins/" "/var/www/www.domain.com/current/web/app/plugins/";
pagespeed LoadFromFile "https://www.domain.com/app/themes/" "/var/www/www.domain.com/current/web/app/themes/";
pagespeed Domain www.domain.com;
pagespeed MapProxyDomain https://www.domain.com/img/ https://img.domain.com/;
# Filters
pagespeed EnableFilters add_instrumentation;
pagespeed EnableFilters convert_gif_to_png,recompress_png;
pagespeed EnableFilters convert_jpeg_to_progressive,recompress_jpeg;
pagespeed EnableFilters convert_jpeg_to_webp,convert_to_webp_lossless,convert_to_webp_animated,recompress_webp;
pagespeed EnableFilters inline_images,resize_images;
pagespeed EnableFilters extend_cache;
pagespeed EnableFilters sprite_images;
pagespeed EnableFilters insert_dns_prefetch;
pagespeed EnableFilters lazyload_images;
pagespeed LazyloadImagesAfterOnload off;
pagespeed EnableFilters inline_css;
pagespeed CssInlineMaxBytes 2048; # 2kb
pagespeed EnableFilters inline_javascript;
pagespeed JsInlineMaxBytes 2048; # 2kb
pagespeed UrlValuedAttribute img data-imgs image;
pagespeed UrlValuedAttribute img data-imgm image;
pagespeed UrlValuedAttribute img data-imgd image;
pagespeed UrlValuedAttribute img data-desktop-src image;
pagespeed UrlValuedAttribute img data-mobile-src image;
pagespeed UrlValuedAttribute img data-mobile-slider-src image;
pagespeed UrlValuedAttribute source srcset image;
# Ensure requests for pagespeed optimized resources go to the pagespeed
# handler and no extraneous headers get set.
location ~ "\.pagespeed\.([a-z]\.)?[a-z]{2}\.[^.]{10}\.[^.]+" { add_header "" ""; }
location ~ "^/ngx_pagespeed_static/" { }
location ~ "^/ngx_pagespeed_beacon" { }
location ~ "^/pagespeed_admin" { allow 127.0.0.1; deny all; }
Hi
But now other files are rewrited? I will say a file in /var/www/www.domain.com/shared/uploads/
is rewrited?
Can you take a look at the log files in img.domain.com? The request from pagespeed are here?
The folder /var/www/www.domain.com/img
exists?
In other hand:
this must be set too
pagespeed UrlValuedAttribute img data-large_image image;
But now other files are rewrited? I will say a file in /var/www/www.domain.com/shared/uploads/ is rewrited?
I'm not 100% sure about uploads, as there is not too may resources loading from that dir and given its high traffic site I can only experiment early in the morning, but resources from other dirs are not processed, eg:
pagespeed LoadFromFile "https://www.domain.com/app/plugins/" "/var/www/www.domain.com/current/web/app/plugins/";
<script defer="" type="text/javascript" src="https://www.domain.com/app/plugins/woocommerce/assets/js/jquery-blockui/jquery.blockUI.min.js?ver=2.7.0-wc.5.6.0" id="jquery-blockui-js"></script>
<!--The preceding resource was not rewritten because its domain (www.domain.com) is not authorized-->
The folder /var/www/www.domain.com/img exists?
There is no img folder both root domain dir and webroot:
In other hand: this must be set too pagespeed UrlValuedAttribute img data-large_image image;
This can be added later.
As I've mentioned everything worked very well for 2-3 years, even after OS migrations there was no issues whatsoever.
From the previous post you can see that pagespeed actually rewrites the img.domain.com image into the www.domain.com/img/ image and its even trying to serve webp image, so I'm wondering if its possible to debug the part of pagespeed fetching the image from img.domain.com and if it's possible to inspect cached resource (download it) or at least get hash / size in bytes?
Hi
<script defer="" type="text/javascript" src="https://www.domain.com/app/plugins/woocommerce/assets/js/jquery-blockui/jquery.blockUI.min.js?ver=2.7.0-wc.5.6.0" id="jquery-blockui-js"></script>
<!--The preceding resource was not rewritten because its domain (www.domain.com) is not authorized-->
This message comes when no pagespeed Domain www.domain.com;
exists, now you have it in place so don´t sould have this.
I put uploads as a example, more exactly: resources in LoadFromFile folders are rewrited?
If this /var/www/www.domain.com/current/web/
is where https://www.domain.com
goes, /var/www/www.domain.com/current/web/img
must exists.
You can take a look at "/dev/shm/ngx_pagespeed_cache"
, here are stored the files cached by pagespeed. Take a look at log files inimg.domain.com
, here you must see request with "Serf" in their User Agent string, even if I remember in the folder img proxied files must be stored.
If there is nothing in these places, some thing is blocking pagespeed fechting the resources from img.domain.com
Can you curl a image from the server that host www.domain.com? Some like running
curl https://img.domain.com/some-folder/some-image.jpg
in the host that have www.domain.com
curl -I https://img.domain.com/image.jpg
(from www.domain.com server)
HTTP/2 200
server: nginx
content-type: image/jpeg
content-length: 21402
cache-control: max-age=31536000, public
date: Fri, 29 Oct 2021 09:24:29 GMT
expires: Sat, 29 Oct 2022 09:24:29 GMT
x-content-type-options: nosniff
It also downloads an image without an issue.
I've looked up logs for img.domain.com, once i purge pagespeed cache i see Serf requests being made:
IP - - [29/Oct/2021:13:03:01 +0300] "GET /95ccb47f-59ed-4ff1-acb8-f41d258a405b/228x228/siemens-ti9573x9rwjpg.jpg HTTP/1.1" 200 12882 "http://www.domain.com/" "Serf/1.3.9 (mod_pagespeed/1.13.35.2-0)"
/dev/shm/ngx_pagespeed_cache directory contain much but /dev/shm/ngx_pagespeed_cache/v3 had domain entry :
Hi So maybe the problem is they are http? Have you these folders and files under https? In chrome developer tools, have you any message in console? (other than the 404 error messages, like some about content-security-policy or about mixed content)
In the log from img.domain.com I can see http://www.domain.com
as referrer so pagespeed is fechting the resource as http not https.
Try pagespeed Domain http*://www.domain.com;
or allmost pagespeed Domain https://www.domain.com;
Try pagespeed Domain http*://www.domain.com; or allmost pagespeed Domain https://www.domain.com;
This did work to an extend, I could see that css/js files have pagespeed hash appended, eg.:
https://www.domain.com/app/plugins/woocommerce/assets/js/frontend/woocommerce.min.js,qver=5.6.0.pagespeed.ce.tywcuxUwoB.js
The resources would return 200, but inspector would show net::ERR_CONTENT_DECODING_FAILED status and it wouldn't load. Quick google search lead me to this https://github.com/apache/incubator-pagespeed-ngx/issues/1346, so I've added pagespeed HttpCacheCompressionLevel 0; and it solved that issue (not sure if this is the right solution?).
Would a need for HttpCacheCompressionLevel indicate that LoadFromFile is not used/working?
The only remaining issue is img.domain.com, those are still not loading returning 404.
Hi HttpCacheCompressionLevel stores optimized resources gziped in pagespeed cache and serve it as gzip with content-encoding: gzip header. If you set HttpCacheCompressionLevel to 0 (gzip disabled) then the resource is stored w/o "gziping" it and served w/o content-encoding header, then the web server need to compress it (whatever compression is enabled) and set the content-encoding header. For example, HttpCacheCompressionLevel must be set to 0 if you will resources be compressed by brotli.
For the img.domain.com issue my bet is pagespeed is requesting resources via http and store it in the http cache, client is requesting the resource via https and there is no https version in the pagespeed cache.
Is http://www.domain.com
accesible? If yes, can you try it an see whats happens with rewrited resources ( 404 or 200)
So the question is why pagespeed is fechting http resources and not https?
Where are the config directives, server block, location block.. ?
pagespeed FetchHttps enable;
pagespeed SslCertDirectory /etc/ssl/certs;
must be set at host or servers blocks not in location ones.
What certificate is in use at img.domain.com, Is a selsigned certificate? is a valid one?
Where are the config directives, server block, location block.. ?
Server block.
What certificate is in use at img.domain.com, Is a selsigned certificate? is a valid one?
Let's encrypt. It is a valid one.
Is http://www.domain.com accesible? There is a 301 redirect to https when accessing over http.
It also seems that pagespeed is attempting to fetch every single file with relative path (e.g. favicon.ico) over http not https.
Hi
I found this old thread https://groups.google.com/g/mod-pagespeed-discuss/c/6g_YjUeDyNU
Can you try pagespeed PreserveUrlRelativity on;
?
https://www.modpagespeed.com/doc/configuration#preserve-url-relativity
Issues with pagespeed were put on hold for some time and I've just took another look at it. The issue still remains with img.domain.com.
Since my last comment, I've moved img.domain.com to completely separate server (it was on the same server as www.domain.com before). Pagespeed is not setup on that server.
www.domain.com the following configuration (recap):
pagespeed on;
pagespeed FetchHttps enable;
pagespeed SslCertDirectory /etc/ssl/certs;
pagespeed Domain https://www.domain.com;
pagespeed MapProxyDomain "https://www.domain.com/img/" "https://img.domain.com";
# Disable version reporting.
pagespeed XHeaderValue "pagespeed";
# Prevent optimizations for selected paths.
pagespeed Disallow "*?wc-api=*";
pagespeed Disallow "*?kdfilter=1*";
pagespeed Disallow "*/wp-json/*";
# Cache
pagespeed FileCachePath /dev/shm/ngx_pagespeed_cache;
pagespeed FileCacheSizeKb 307200; # 300mb
pagespeed HttpCacheCompressionLevel 0;
# Admin console
pagespeed Statistics on;
pagespeed StatisticsLogging on;
pagespeed LogDir /var/log/pagespeed;
pagespeed AdminPath /pagespeed_admin;
pagespeed EnableCachePurge on;
pagespeed PreserveUrlRelativity on;
location ~ "^/pagespeed_admin" {
allow 127.0.0.1;
deny all;
}
# Ensure requests for pagespeed optimized resources go to the pagespeed
# handler and no extraneous headers get set.
location ~ "\.pagespeed\.([a-z]\.)?[a-z]{2}\.[^.]{10}\.[^.]+" {
add_header "" "";
}
location ~ "^/ngx_pagespeed_static/" { }
location ~ "^/ngx_pagespeed_beacon" { }
pagespeed RewriteLevel PassThrough;
pagespeed EnableFilters add_instrumentation;
pagespeed EnableFilters extend_cache;
pagespeed DisableRewriteOnNoTransform off;
pagespeed EnableFilters insert_dns_prefetch;
pagespeed EnableFilters inline_javascript;
pagespeed JsInlineMaxBytes 2048; # 2kb
pagespeed EnableFilters inline_css;
pagespeed CssInlineMaxBytes 2048; # 2kb
pagespeed EnableFilters convert_gif_to_png,recompress_png;
pagespeed EnableFilters convert_jpeg_to_progressive,recompress_jpeg;
pagespeed EnableFilters convert_jpeg_to_webp,convert_to_webp_lossless,convert_to_webp_animated,recompress_webp;
pagespeed EnableFilters inline_images,resize_images;
pagespeed EnableFilters sprite_images;
pagespeed EnableFilters lazyload_images;
pagespeed LazyloadImagesAfterOnload off;
Not sure of this of any help but this is typical with all img.domain.com files:
[Fri, 01 Apr 2022 12:09:16 GMT] [Info] [668435] <...> Shrinking image `https://img.domain.com/3b68450e-44ba-4f7d-a7b3-dff8178eb5c9/250x250/352195jpg.jpg' (19599 bytes) to `https://www.domain.com/img/3b68450e-44ba-4f7d-a7b3-dff8178eb5c9/250x250/x352195jpg.jpg.pagespeed.ic.FnOL3MV8jb.webp' (7202 bytes)
[Fri, 01 Apr 2022 12:09:33 GMT] [Warning] [668435] [x352195jpg.jpg:0] Resource based on http://www.domain.com/img/3b68450e-44ba-4f7d-a7b3-dff8178eb5c9/250x250/352195jpg.jpg but cannot access the original
I understand that first log line says that pagespeed managed to download and compress the file, I can also see a file present in /dev/shm/ngx_pagespeed_cache/v3/domain.com/https,3A/,2Fwww.domain.com/img/c4cfe5b0-1373-4bbf-86d2-39921659dd9b/250x250:
-rw------- 1 www-data www-data 7410 bal. 1 15:09 x3493651jpg.jpg.pagespeed.ic.JqkCRV6dWY.webp,
The second log line is generated after subsequent page load, key observations:
-rw------- 1 www-data www-data 173 bal. 1 15:42 352195jpg.jpg,
It seems to me that the pagespeed downloads the file, creates webp but cached file is not loaded on subsequent requests.
Hi
Maybe you need to create the proxied folders? I mean /var/www/domain-com/img/
A far time ago I have tried this, create the directory for the proxied assets and the directive are like:
pagespeed MapProxyDomain "https://www.domain.com/img/" "https://img.domain.com/";
Note the last /.
But maybe the 404 comes because pagespeed search the assets in https cache and it is stored in http. Do you have a /dev/shm/ngx_pagespeed_cache/v3/domain.com/https folder? What do you have in the img.domain.com logs? Is pagespeed fechting the resources as http? In the html code the images are like src="//img.domain.com/some.image.jpg"? can you try to add the protocol, https?
And maybe you can try:
pagespeed MapOriginDomain http://img.domain.com/ https://img.domain.com/ pagespeed MapRewriteDomain https://www.domain.com/img/ https://img.domain.com/ pagespeed MapProxyDomain http://www.domain.com/ http://img.domain.com
Maybe this config need
pagespeed Domain http*://www.domain.com/ to set both http and https domains.
Maybe you need to create the proxied folders? I mean /var/www/domain-com/img/
I've tried that as well, no observable difference.
In the html code the images are like src="//img.domain.com/some.image.jpg"? can you try to add the protocol, https?
It's always with https://
And maybe you can try:
pagespeed MapOriginDomain http://img.domain.com/ https://img.domain.com/
pagespeed MapRewriteDomain https://www.domain.com/img/ https://img.domain.com/
pagespeed MapProxyDomain http://www.domain.com/ http://img.domain.com/
Maybe this config need
pagespeed Domain http*://www.domain.com/ to set both http and https domains.
With ?PageSpeedFilters=+debug I see the debug messages like this for all img.domain.com images:
Uncacheable content, preventing rewriting of https://img.domain.com/0416425d-81f4-4f48-ab83-c3dc7a1a2e39/228x228/de65085msjpg.jpg
Note: http traffic is always redirected:
server {
listen 80;
server_name www.domain.com;
location /.well-known/acme-challenge/ {
root /var/www/letsencrypt.challenges;
}
location / {
return 301 https://$host$request_uri;
}
}
Hi
Uncacheable content, preventing rewriting of https://img.domain.com/0416425d-81f4-4f48-ab83-c3dc7a1a2e39/228x228/de65085msjpg.jpg
Some header is preventing the resource to be cached. For example a header like Vary: Cookie
prevents caching the resource.
What are the whole headers when you fecth the image from the nginx server?
HTTP/1.1 200 OK Server: nginx Content-Type: image/jpeg Content-Length: 12183 Connection: keep-alive Cache-Control: max-age=31536000, public Date: Mon, 04 Apr 2022 09:43:57 GMT Expires: Tue, 04 Apr 2023 09:43:57 GMT X-Content-Type-Options: nosniff
Hi
Well, these headers are ok for caching the resource, but pagespeed found some reason to don´t cache it.
If you put pagespeed off and don´t rewrite img.domain.com to www.domain.com/img/ the image is loaded?
Are here any CSP in www.domain.com?
Have you tried this?:
pagespeed MapOriginDomain http://img.domain.com/ https://img.domain.com/
pagespeed MapRewriteDomain https://www.domain.com/img/ https://img.domain.com/
pagespeed MapProxyDomain http://www.domain.com/ http://img.domain.com/
This need http://img.domain.com be reachable.
For me sound strange that the rewrited resource is stored in the http cache and not in the https one as you say here:
There is a file present in /dev/shm/ngx_pagespeed_cache/v3/domain.com/http,3A/,2Fwww.domain.com/img/3b68450e-44ba-4f7d-a7b3-dff8178eb5c9/250x250:
The uncacheable message comes from here
In the pagespeed admin pages -> Message History there is any error related to these images? Maybe a 10002 or 10003 error?
I've been running ngx_pagespeed 1.13.35.2-stable on Ubuntu 16.04 and now on Ubuntu 20.04 without any issues until this morning when I've noticed that every single image served from img.domain.com returns 404. Interesting bit there is that pagespeed replaced the image urls and trying to serve cached webp image, but cached image returns 404. CSS/JS optmizations stopped working too.
I've tried restarting nginx and purging the pagespeed cache through pagespeed_admin without a success.
I've enabled MessageBuffer but there is only info messages and it does not give any additional info why this might be happening. /var/log/pagespeed seems to be useless (statistics?) and I'm not aware of any additional logs for ngx_pagespeed.
I'm not sure if this in any way related, but dpkg log indicated that this morning there was some package upgrades, including ca-certificates:
ca-certificates (Unpacking ca-certificates (20210119~20.04.2) over (20210119~20.04.1) ...)
Here is the pagespeed configuration for a single site (there was no changes recently):