search

apache / incubator-pagespeed-ngx

Automatic PageSpeed optimization module for Nginx
http://ngxpagespeed.com/
Apache License 2.0
4.37k stars 362 forks source link

URL is not in cache #1720

Open uhlhosting opened 3 years ago

uhlhosting commented 3 years ago

Hi,

After deploying pagespeed on my server, tested several configs and options, yet lately these info messages are filling the console, was wondering if anyone knows how to handle these:

Attached the console messages in a gist. https://gist.github.com/uhlhosting/c6bf91d643b8fc0d86f6459eddb80e4c

Running on CentOS 8, with nginx 1.19.4.

Version: 14: on

Filters
ah  Add Head
cw  Collapse Whitespace
cc  Combine Css
jc  Combine Javascript
gp  Convert Gif to Png
jp  Convert Jpeg to Progressive
jw  Convert Jpeg To Webp
mc  Convert Meta Tagsa
pj  Convert Png to Jpeg
ws  When converting images to WebP, prefer lossless conversions
ec  Cache Extend Css
ei  Cache Extend Images
es  Cache Extend Scripts
fc  Fallback Rewrite Css 
if  Flatten CSS Imports
hw  Flushes html
ci  Inline Css
ii  Inline Images
il  Inline @import to Link
ji  Inline Javascript
js  Jpeg Subsampling
rj  Recompress Jpeg
rp  Recompress Png
rw  Recompress Webp
rc  Remove Comments
ri  Resize Images
cf  Rewrite Css
jm  Rewrite External Javascript
jj  Rewrite Inline Javascript
cu  Rewrite Style Attributes With Url
cp  Strip Image Color Profiles
md  Strip Image Meta Data

Options
  AdminPath (nap)                              /pagespeed_admin
  AvoidRenamingIntrospectiveJavascript (aris)  True
  BeaconUrl (bu)                               /ngx_pagespeed_beacon
  ConsolePath (ncp)                            /pagespeed_console
  CriticalImagesBeaconEnabled (cibe)           True
  EnableCachePurge (euci)                      True
  EnableRewriting (e)                          1
  FetchHttps (fhs)                             enable
  FileCacheCleanIntervalMs (afcci)             3600000
  FileCacheInodeLimit (afcl)                   500000
  FileCachePath (afcp)                         /mnt/ram/pagespeed
  FileCacheSizeKb (afc)                        102400
  GlobalAdminPath (ngap)                       /pagespeed_global_admin
  GlobalStatisticsPath (ngsp)                  /ngx_pagespeed_global_statistics
  HttpCacheCompressionLevel (hccl)             0
  ImageMaxRewritesAtOnce (im)                  3
  LogDir (ald)                                 /var/log/pagespeed
  LowercaseHtmlNames (lh)                      True
  MaxSegmentLength (uss)                       250
  MemcachedServers (ams)                       127.0.0.1:11211
  MemcachedTimeoutUs (amo)                     1000000
  MessagesPath (nmp)                           /ngx_pagespeed_message
  PreserveUrlRelativity (pur)                  True
  PrivateNotVaryForIE (pnvie)                  False
  RespectXForwardedProto (rxfp)                True
  RewriteLevel (l)                             Core Filters
  RewriteRandomDropPercentage (rrdp)           1
  Statistics (ase)                             True
  StatisticsLogging (asle)                     True
  StatisticsPath (nsp)                         /ngx_pagespeed_statistics

Domain Lawyer
  http://uhl.site/ Auth RewriteDomain:http://www.uhl.site/
  http://www.uhl.site/ Auth
  https://uhl.site/ Auth OriginDomain:http://uhl.site/

Invalidation Timestamp: (none)
Lofesa commented 3 years ago

Hi These messages are normal, as far as pagespeed don´t cache the htlm code: Could not rewrite resource in-place because URL is not in cache: https://uhl.site/ These others not:

Trying to serve rewritten resource in-place: http://127.0.0.1/nginx_status
Could not rewrite resource in-place because URL is not in cache: http://127.0.0.1/nginx_status

What are yours directives:

pagespeed MapRewriteDomain
pagespeed Domain
pagespeed MapOriginDomain

How your site works? I will say: Are served from http localhost and maped to https://uhl.site/, have a proxy cache?.... What type of cache are in place?

uhlhosting commented 3 years ago

Hi, Using Nginx + FastCGI cache + opcache + redis this was the main original stack.

I then added pagespeed and testing since, if its needed will remove the previous ones.

https://gist.github.com/uhlhosting/4ef7e41ece8ee0fc2ab8fa6049766b78 here is the uhl.site config. Here is the nginx.conf https://gist.github.com/uhlhosting/634ce18b5f30f9dcb22e931ccc500ecc

About uhl.site true since not caching html, alto not sure why it toke the stub url address for status from nginx into connsideration.

Lofesa commented 3 years ago

Hi Some questions about the config files:

1.- There is some files (pagespeed.conf in file uhl.site.confline 5, pagespeedstatslog.conf line 68, pagespeedhandler.confline 69) you have no included. Maybe in this files are the directives for nginx_status and pagespeed log or the server block for 127.0.0.1?

2.- Why you have fastcgi directives out of a server block in uhl.site.conf? If the directives are out of a server block is the same as put it in the http block, so why don´t put it at the same level that other fastcgi directives in nginx.conf

3.- You have enbled pagespeed in the http block so pagespeed works for all the server blocks. You need pagespeed working in phpadmin or in the default server? if not, put pagespeed unplugged; in it to disable pagespeed in these servers

4.- Why you need pagespeed MapOriginDomain "http://uhl.site" "https://uhl.site";? You serve uhl.site in https protocol and when http://uhl.site is requested, is redirected to https://uhl.site (as per the uhl.site.conf config)

5.- Why you need if ($scheme = http)? If the request go to uhl.site port 80, the request is allways http scheme, I think the if is not needed and the request must be inconditionally redirected to uhl.site port 443.

6.- Why do you need pagespeed ModPagespeedRespectXForwardedProto on;? This directive is used when you have a ssl endpoint before the server: user -> https proxy/ssl endpont -> http server and if needed, because you have a ssl endoint, you need pagespeed RespectXForwardedProto on; (for nginx. You have used the syntax for apache)

7.- What means this pagespeed Allow wildcard_spec;? Here wildcard_spec here is a regular expresion, not the this literal text. By default (if these directive is NOT present) the value is pagespeed Allow "*"; so maybe you don´t need it. See the examples here

8.- You have pagespeed ModPagespeedFetchHttps enable; this directive is for apache, for nginx is pagespeed FetchHttps enable;

9.- You say use Redis, but in the config files Memcached is configured, not Redis.

10.- Why you need gzip enabled in .pagespeed. reosurces?

11.- If you use wordpress, maybe you need to exclude wp-admin from pagespeed: pagespeed Disallow /wp-admin;

12.- Why you use default_type application/octet-stream;?

uhlhosting commented 3 years ago

Hi,

Here are the remaining details:

./pagespeed.conf:pagespeed MessageBufferSize 100000;
./pagespeed.conf:pagespeed NumRewriteThreads 6;
./pagespeed.conf:pagespeed NumExpensiveRewriteThreads 6;
./pagespeed.conf:pagespeed ImageMaxRewritesAtOnce 3;
./pagespeed.conf:pagespeed LogDir /var/log/pagespeed;
./pagespeed.conf:pagespeed FileCachePath /mnt/ram/pagespeed;
./pagespeed.conf:pagespeed StatisticsPath /ngx_pagespeed_statistics;
./pagespeed.conf:pagespeed GlobalStatisticsPath /ngx_pagespeed_global_statistics;
./pagespeed.conf:pagespeed MessagesPath /ngx_pagespeed_message;
./pagespeed.conf:pagespeed ConsolePath /pagespeed_console;
./pagespeed.conf:pagespeed AdminPath /pagespeed_admin;
./pagespeed.conf:pagespeed GlobalAdminPath /pagespeed_global_admin;
./pagespeedhandler.conf:#  Ensure requests for pagespeed optimized resources go to the pagespeed
./pagespeedhandler.conf:location ~ "\.pagespeed\.([a-z]\.)?[a-z]{2}\.[^.]{10}\.[^.]+" { add_header "" ""; }
./pagespeedhandler.conf:location ~ "^/pagespeed_static/" { }
./pagespeedhandler.conf:location ~ "^/ngx_pagespeed_beacon$" { }
./pagespeedhandler.conf:location /ngx_pagespeed_statistics { include custom/admin-ips.conf; deny all; }
./pagespeedhandler.conf:location /ngx_pagespeed_global_statistics { include custom/admin-ips.conf; deny all; }
./pagespeedhandler.conf:location /ngx_pagespeed_message { include custom/admin-ips.conf; deny all; }
./pagespeedhandler.conf:location /pagespeed_console { include custom/admin-ips.conf; deny all; }
./pagespeedhandler.conf:location ~ ^/pagespeed_admin { include custom/admin-ips.conf; deny all; }
./pagespeedhandler.conf:location ~ ^/pagespeed_global_admin { include custom/admin-ips.conf; deny all; }
./pagespeedstatslog.conf:pagespeed Statistics on;
./pagespeedstatslog.conf:pagespeed StatisticsLogging on;
./pagespeedstatslog.conf:pagespeed EnableCachePurge on;

  1. At the time of setup, well i had issues, and asked around since we test aapanel, so according to their docs, thats the way to enable fastcgi, based on this nginx configuration we use.

  2. I was testing this option also with unplugged, guess will just try it and in end will leave only main websites.

  3. This indeed has no meaning, was just testing to see if this works as a redirect or if it helps.

  4. Thats another default setting that come with our aapanel, its in their template, for the time beying was not causing me loops or issues, so i left it.

  5. I have to review the settings, while this is fresh deployment, surely had missplaced from apache directives instead of nginx ones, will double check and fix.

  6. yes you are right, il change this to * .

  7. will update this.

  8. I said redis on the server side, with wordpress. Indeed ngx has memcached, since redis was experimental accoring to docs.

  9. I found some online threads using it, as a test just gave this a chance.

11.I had some dissalow rules in wordpress i first config, will introduce them again. Needed a bit more research.

  1. no specific reason, it was in by default, i left it be, will tidy up config.

Beside this, I noticed several things, my security headers seems to be ignored some of them.

Also the caching time for all ressources has changed now, and is mixed some ressourced cached 5 mins, while some 30 days.

Lofesa commented 3 years ago

Hi 1.- these 3 files have a mixed directives content. Some are needed for the server block you enable pagespeed or can be set at http block, but other are needed only in the server block you use to view the console and stats pages. And for the nignx logic all tree files must be set in a server block (you have include pagespeed.conf; in the http block). Niginx uses this:

http{
      server {
            location{
            }
       }
}

Maybe a reorder of these files is needed, making all directives for statistics, console and other in 1 file for each, and not spreaded in these 3.

2.- All these fastcgi are used ONLY where you have php locations, so you can put it together and have all fascgi directives only in one point and not spread it.

3.- Personally I use only pagespeed on; ony in the server blocks I need it, and a explicit pagespeed unplugged; in those I don´t need.

4.- I think you don´t need pagespeed MapOriginDomain "http://uhl.site" "https://uhl.site"; at all.

5.- this if is not needed, like if ($server_port !~ 443) (in uhl.site.conf lines 77 - 80). These directives make sense when you have a server block with ports 80 and 443 enabled, but you have these ports in diferent server blocks so waste resources making a test not need. It works? sure, but you have not enabled https in the 80 port so a https never reach this server, and you have only enabled the 443 in the server block thus a request to others port never reach this.

9.- You can use Redis. In the docs say "experimental" but don´t is. Is safe and estable, and for me better option that memcached.

10.- You have brotli and gzip enabled in the http block. and these are applied to .pagespeed. resources too.

11.- Pagespeed may have bad consecuences in the admin area for wordpress. Better disabled for it. Same for the fastcgi cache, you don´t need the wp-login cached.

12.- You are not serving binary content by default but text/html, better change it.

13.- I see you have an other include directive i´m not aware before: include proxy.conf;. I can´t see any config directive that say you are using a proxy, so maybe the directives in this files introduces some distortion on how the site works. And one more: include include enable-php.conf; or include /www/server/panel/vhost/rewrite/uhl.site.conf;

Maybe I´m lossing some info with all those includes...... Maybe the panel you use have cnfigured nginx to work with a proxy, then it may work as:

user -> nginx proxy -> nginx server

and then some directives make sense.

uhlhosting commented 3 years ago 
proxy_temp_path /www/server/nginx/proxy_temp_dir;
proxy_cache_path /www/server/nginx/proxy_cache_dir levels=1:2 keys_zone=cache_one:20m inactive=1d max_size=5g;
client_body_buffer_size 512k;
proxy_connect_timeout 60;
proxy_read_timeout 60;
proxy_send_timeout 60;
proxy_buffer_size 32k;
proxy_buffers 4 64k;
proxy_busy_buffers_size 128k;
proxy_temp_file_write_size 128k;
proxy_next_upstream error timeout invalid_header http_500 http_503 http_404;
proxy_cache cache_one;

This is the proxy.conf.

    location ~ [^/]\.php(/|$)
    {
        try_files $uri =404;
        fastcgi_pass  unix:/tmp/php-cgi-74.sock;
        fastcgi_index index.php;
        include fastcgi.conf;
        include pathinfo.conf;
    }

This it the enable-php.conf.

the rewrite file is empty yet that is linked to the frontend where users can add manually rewrite rules, at the moment is empty.

  1. here that was correct your mention of port 80, i had it together before, just some of my testing made me change it, i changed it back so now its on same server entry with 443 this was the default .

Other very strange behaviour would be the page sometimes giving a blank white page, and also there is a 404 and a 501 errors hit from time to time. guess still experimenting , would need to start with less filters.

I will update the newly renewed configs, once I finish them. Any idea why the timeouts of the domain when using refresh only.

Lofesa commented 3 years ago

To debug the blank page you need to look at server logs and see what happens when the blank page come. Maybe you need to put error messages in debug level to get more info: error_log /path/to/log debug;

I think that the panel had created very complicated config file structure for a simple one server config. Maybe you need some step by step process to get a whole config, I mean, make the simplified setup config as posible w/o proxy or even pagespeed, make it work clean and w/o any errors, and then intrude things like the pagespeed module.

uhlhosting commented 3 years ago

Seems that is not helping, seems no errors are recorder whatsoever:

94.112.41.18 - - [23/Nov/2020:11:09:02 +0100] "GET / HTTP/2.0" 200 10339 "https://uhl.site/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.1 Safari/605.1.15"
94.112.41.18 - - [23/Nov/2020:11:09:02 +0100] "GET /wp-content/uploads/2020/11/Testimonial-Background,402x.png.pagespeed.ce.3PAmB2kjHN.png HTTP/2.0" 200 24886 "https://uhl.site/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.1 Safari/605.1.15"
94.112.41.18 - - [23/Nov/2020:11:09:02 +0100] "GET /wp-content/uploads/2020/11/Footer-Background,402x.png.pagespeed.ce.jADOP1tFTI.png HTTP/2.0" 200 27969 "https://uhl.site/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko)

This is the most we have from access log, while error log is empty on the vhost side.

uhlhosting commented 3 years ago

Seems that even debug does not track it, there is a bug maybe?

The most intresting part is that error console from nginx in the site vhost is not showing any errors. And the browser registers a 200 page on hit, yet the page is blank. This upon refresh clears out from time to time.

Managed to get something in error log:

2020/11/23 12:02:28 [notice] 3633495#0: *463 "443" matches "443", client: 94.112.41.18, server: uhl.site, request: "GET / HTTP/2.0", host: "uhl.site", referrer: "https://uhl.site/hello-world/"
2020/11/23 12:02:28 [notice] 3633495#0: *463 "/wp-admin/|/xmlrpc.php|wp-.*.php|/feed/|index.php|sitemap(_index)?.xml" does not match "/", client: 94.112.41.18, server: uhl.site, request: "GET / HTTP/2.0", host: "uhl.site", referrer: "https://uhl.site/hello-world/"
2020/11/23 12:02:28 [notice] 3633495#0: *463 "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_no_cache|wordpress_logged_in" does not match "X_CACHE_KEY=29a7cb29daceb7c27b6294402d720a1a", client: 94.112.41.18, server: uhl.site, request: "GET / HTTP/2.0", host: "uhl.site", referrer: "https://uhl.site/hello-world/"
2020/11/23 12:02:28 [notice] 3633495#0: *463 "/wp-admin$" does not match "/", client: 94.112.41.18, server: uhl.site, request: "GET / HTTP/2.0", host: "uhl.site", referrer: "https://uhl.site/hello-world/"
2020/11/23 12:02:28 [notice] 3633495#0: *463 "443" matches "443", client: 94.112.41.18, server: uhl.site, request: "GET / HTTP/2.0", host: "uhl.site", referrer: "https://uhl.site/hello-world/"
2020/11/23 12:02:28 [notice] 3633495#0: *463 "/wp-admin/|/xmlrpc.php|wp-.*.php|/feed/|index.php|sitemap(_index)?.xml" does not match "/", client: 94.112.41.18, server: uhl.site, request: "GET / HTTP/2.0", host: "uhl.site", referrer: "https://uhl.site/hello-world/"
2020/11/23 12:02:28 [notice] 3633495#0: *463 "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_no_cache|wordpress_logged_in" does not match "X_CACHE_KEY=29a7cb29daceb7c27b6294402d720a1a", client: 94.112.41.18, server: uhl.site, request: "GET / HTTP/2.0", host: "uhl.site", referrer: "https://uhl.site/hello-world/"
2020/11/23 12:02:28 [notice] 3633495#0: *463 "/wp-admin$" does not match "/", client: 94.112.41.18, server: uhl.site, request: "GET / HTTP/2.0", host: "uhl.site", referrer: "https://uhl.site/hello-world/"
2020/11/23 12:02:28 [info] 3633495#0: [ngx_pagespeed 1.13.35.2-0] Trying to serve rewritten resource in-place: https://uhl.site/
2020/11/23 12:02:28 [info] 3633495#0: [ngx_pagespeed 1.13.35.2-0] Cache entry is expired: https://uhl.site/ (fragment=uhl.site)
2020/11/23 12:02:28 [info] 3633495#0: [ngx_pagespeed 1.13.35.2-0] Could not rewrite resource in-place because URL is not in cache: https://uhl.site/
2020/11/23 12:02:28 [notice] 3633495#0: *463 "443" matches "443", client: 94.112.41.18, server: uhl.site, request: "GET / HTTP/2.0", host: "uhl.site", referrer: "https://uhl.site/hello-world/"
2020/11/23 12:02:28 [notice] 3633495#0: *463 "/wp-admin/|/xmlrpc.php|wp-.*.php|/feed/|index.php|sitemap(_index)?.xml" does not match "/", client: 94.112.41.18, server: uhl.site, request: "GET / HTTP/2.0", host: "uhl.site", referrer: "https://uhl.site/hello-world/"
2020/11/23 12:02:28 [notice] 3633495#0: *463 "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_no_cache|wordpress_logged_in" does not match "X_CACHE_KEY=29a7cb29daceb7c27b6294402d720a1a", client: 94.112.41.18, server: uhl.site, request: "GET / HTTP/2.0", host: "uhl.site", referrer: "https://uhl.site/hello-world/"
2020/11/23 12:02:28 [notice] 3633495#0: *463 "/wp-admin$" does not match "/index.php", client: 94.112.41.18, server: uhl.site, request: "GET / HTTP/2.0", host: "uhl.site", referrer: "https://uhl.site/hello-world/"
2020/11/23 12:02:28 [info] 3633495#0: [ngx_pagespeed 1.13.35.2-0] Trying to serve rewritten resource in-place: https://uhl.site/
2020/11/23 12:02:28 [info] 3633495#0: [ngx_pagespeed 1.13.35.2-0] Cache entry is expired: https://uhl.site/ (fragment=uhl.site)
2020/11/23 12:02:28 [info] 3633495#0: [ngx_pagespeed 1.13.35.2-0] Could not rewrite resource in-place because URL is not in cache: https://uhl.site/
2020/11/23 12:02:28 [info] 3633495#0: [ngx_pagespeed 1.13.35.2-0] Cache entry is expired: https://uhl.site/ (fragment=uhl.site)
2020/11/23 12:02:28 [notice] 3633495#0: *464 "443" matches "443", client: 172.18.6.10, server: uhl.site, request: "GET / HTTP/1.1", host: "uhl.site", referrer: "https://uhl.site/"
2020/11/23 12:02:28 [notice] 3633495#0: *464 "/wp-admin/|/xmlrpc.php|wp-.*.php|/feed/|index.php|sitemap(_index)?.xml" does not match "/", client: 172.18.6.10, server: uhl.site, request: "GET / HTTP/1.1", host: "uhl.site", referrer: "https://uhl.site/"
2020/11/23 12:02:28 [notice] 3633495#0: *464 "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_no_cache|wordpress_logged_in" does not match "", client: 172.18.6.10, server: uhl.site, request: "GET / HTTP/1.1", host: "uhl.site", referrer: "https://uhl.site/"
2020/11/23 12:02:28 [notice] 3633495#0: *464 "/wp-admin$" does not match "/", client: 172.18.6.10, server: uhl.site, request: "GET / HTTP/1.1", host: "uhl.site", referrer: "https://uhl.site/"
2020/11/23 12:02:28 [debug] 3633495#0: *463 CollectAccumulatedWrites, -2
2020/11/23 12:02:28 [debug] 3633495#0: *463 CollectAccumulatedWrites, -2
2020/11/23 12:02:28 [debug] 3633495#0: *463 CollectAccumulatedWrites, 0
2020/11/23 12:02:28 [info] 3633495#0: *464 client 172.18.6.10 closed keepalive connection
2020/11/23 12:02:28 [info] 3633495#0: [ngx_pagespeed 1.13.35.2-0] HTTPCache key=https://uhl.site/ fragment=uhl.site: remembering recent failure for 299 seconds.
2020/11/23 12:02:28 [notice] 3633495#0: *463 "443" matches "443", client: 94.112.41.18, server: uhl.site, request: "GET / HTTP/2.0", host: "uhl.site", referrer: "https://uhl.site/"
2020/11/23 12:02:28 [notice] 3633495#0: *463 "/wp-admin/|/xmlrpc.php|wp-.*.php|/feed/|index.php|sitemap(_index)?.xml" does not match "/", client: 94.112.41.18, server: uhl.site, request: "GET / HTTP/2.0", host: "uhl.site", referrer: "https://uhl.site/"
2020/11/23 12:02:28 [notice] 3633495#0: *463 "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_no_cache|wordpress_logged_in" does not match "X_CACHE_KEY=29a7cb29daceb7c27b6294402d720a1a", client: 94.112.41.18, server: uhl.site, request: "GET / HTTP/2.0", host: "uhl.site", referrer: "https://uhl.site/"
2020/11/23 12:02:28 [notice] 3633495#0: *463 "/wp-admin$" does not match "/", client: 94.112.41.18, server: uhl.site, request: "GET / HTTP/2.0", host: "uhl.site", referrer: "https://uhl.site/"
Lofesa commented 3 years ago

Hi.... Occam, a missconfiguration rather tan a bug. For example, in you fastcgi.conf have you a line like: fastcgi_param SCRIPT_FILENAME $document_root/$fastcgi_script_name; or in the uhl.site.conf there is this at line 101: try_files $uri =404; this try to find a file that match the uri and if not found return a 404. Better try to use this try_files $uri $uri/ /index.php$is_args$args ; This tries to fin a file, if not found, try if is a folder, if not found try the index.php with parameters (if one is set) In wordpress if you use pretty url, some like https://uhl.site/some-post/ the /some-post/ part is not a file nor folder but rather than wordpress convert it in some like https://uhl.site/index.php?post-id=XXXX

uhlhosting commented 3 years ago

Changed that line:

2020/11/23 12:21:08 [info] 3638298#0: [ngx_pagespeed 1.13.35.2-0] HTTPCache key=https://uhl.site/ fragment=uhl.site: remembering recent failure for 283 seconds.
2020/11/23 12:21:08 [debug] 3638298#0: *204 CollectAccumulatedWrites, 0
2020/11/23 12:21:09 [notice] 3638298#0: *199 "443" matches "443", client: 94.112.41.18, server: uhl.site, request: "GET /hello-world/ HTTP/2.0", host: "uhl.site", referrer: "https://uhl.site/"
2020/11/23 12:21:09 [notice] 3638298#0: *199 "/wp-admin/|/xmlrpc.php|wp-.*.php|/feed/|index.php|sitemap(_index)?.xml" does not match "/hello-world/", client: 94.112.41.18, server: uhl.site, request: "GET /hello-world/ HTTP/2.0", host: "uhl.site", referrer: "https://uhl.site/"
2020/11/23 12:21:09 [notice] 3638298#0: *199 "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_no_cache|wordpress_logged_in" does not match "wp-settings-1=libraryContent%3Dbrowse; wp-settings-time-1=1605847282; X_CACHE_KEY=ac0b86989d4a9ef403a39cb4ee779433", client: 94.112.41.18, server: uhl.site, request: "GET /hello-world/ HTTP/2.0", host: "uhl.site", referrer: "https://uhl.site/"
2020/11/23 12:21:09 [notice] 3638298#0: *199 "/wp-admin$" does not match "/hello-world/", client: 94.112.41.18, server: uhl.site, request: "GET /hello-world/ HTTP/2.0", host: "uhl.site", referrer: "https://uhl.site/"

these were not previously, will check also to the admins from aapanel what they say about the config.

Lofesa commented 3 years ago

Try this config. it is generated by this page:

https://www.digitalocean.com/community/tools/nginx?global.security.limitReq=true&global.php.phpServer=%2Fvar%2Frun%2Fphp%2Fphp7.4-fpm.sock&global.performance.brotliCompression=true&global.nginx.user=%20www

with the data I know for your site, maybe need some modifications.

nginxconfig.io-uhl.site.tar.gz

uhlhosting commented 3 years ago

@Lofesa thanks il give it a try, anyhow that would be more complex to addapt to the deployment of nginx we have now.

I have found now a recent docker deployment and im testing it out.

It seems when I disable pagespeed the white pages are out, so il start slowly taking all issues one by one. Il also try to involve the aapanel admins.

uhlhosting commented 3 years ago

It seems there is some conflicts of some sort, while in the moment I enabled pagespeed, imediatelly my pages goes from 404 to 501 and back and forth to white screen.

image image

While access console shows:

94.112.41.18 - - [23/Nov/2020:20:35:13 +0100] "GET / HTTP/2.0" 404 1 "https://uhl.site/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4327.0 Safari/537.36" "-" "uhl.site" sn="uhl.site" rt=0.001 ua="-" us="-" ut="-" ul="-" cs=-
94.112.41.18 - - [23/Nov/2020:20:35:14 +0100] "GET / HTTP/2.0" 200 1 "https://uhl.site/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4327.0 Safari/537.36" "-" "uhl.site" sn="uhl.site" rt=0.000 ua="-" us="-" ut="-" ul="-" cs=-
94.112.41.18 - - [23/Nov/2020:20:35:14 +0100] "GET /favicon.ico HTTP/2.0" 302 2150 "https://uhl.site/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4327.0 Safari/537.36" "-" "uhl.site" sn="uhl.site" rt=0.002 ua="-" us="-" ut="-" ul="-" cs=HIT
94.112.41.18 - - [23/Nov/2020:20:35:14 +0100] "GET /wp-content/uploads/2020/11/cropped-web-programming-32x32.png HTTP/2.0" 200 560 "https://uhl.site/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4327.0 Safari/537.36" "-" "uhl.site" sn="uhl.site" rt=0.001 ua="-" us="-" ut="-" ul="-" cs=-
94.112.41.18 - - [23/Nov/2020:20:35:15 +0100] "GET / HTTP/2.0" 200 1 "https://uhl.site/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4327.0 Safari/537.36" "-" "uhl.site" sn="uhl.site" rt=0.000 ua="-" us="-" ut="-" ul="-" cs=-
94.112.41.18 - - [23/Nov/2020:20:35:15 +0100] "GET / HTTP/2.0" 200 1 "https://uhl.site/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4327.0 Safari/537.36" "-" "uhl.site" sn="uhl.site" rt=0.000 ua="-" us="-" ut="-" ul="-" cs=-
94.112.41.18 - - [23/Nov/2020:20:35:15 +0100] "GET /favicon.ico HTTP/2.0" 302 2150 "https://uhl.site/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4327.0 Safari/537.36" "-" "uhl.site" sn="uhl.site" rt=0.001 ua="-" us="-" ut="-" ul="-" cs=HIT
94.112.41.18 - - [23/Nov/2020:20:35:16 +0100] "GET /favicon.ico HTTP/2.0" 302 2150 "https://uhl.site/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4327.0 Safari/537.36" "-" "uhl.site" sn="uhl.site" rt=0.001 ua="-" us="-" ut="-" ul="-" cs=HIT
94.112.41.18 - - [23/Nov/2020:20:35:16 +0100] "GET / HTTP/2.0" 499 0 "https://uhl.site/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4327.0 Safari/537.36" "-" "uhl.site" sn="uhl.site" rt=0.000 ua="-" us="-" ut="-" ul="-" cs=-
94.112.41.18 - - [23/Nov/2020:20:35:16 +0100] "GET / HTTP/2.0" 200 1 "https://uhl.site/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4327.0 Safari/537.36" "-" "uhl.site" sn="uhl.site" rt=0.000 ua="-" us="-" ut="-" ul="-" cs=-
94.112.41.18 - - [23/Nov/2020:20:35:16 +0100] "GET /wp-content/uploads/2020/11/cropped-web-programming-32x32.png HTTP/2.0" 200 560 "https://uhl.site/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4327.0 Safari/537.36" "-" "uhl.site" sn="uhl.site" rt=0.001 ua="-" us="-" ut="-" ul="-" cs=-
94.112.41.18 - - [23/Nov/2020:20:35:17 +0100] "GET /favicon.ico HTTP/2.0" 302 2150 "https://uhl.site/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4327.0 Safari/537.36" "-" "uhl.site" sn="uhl.site" rt=0.002 ua="-" us="-" ut="-" ul="-" cs=HIT
94.112.41.18 - - [23/Nov/2020:20:35:17 +0100] "GET /wp-content/uploads/2020/11/cropped-web-programming-32x32.png HTTP/2.0" 200 560 "https://uhl.site/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4327.0 Safari/537.36" "-" "uhl.site" sn="uhl.site" rt=0.001 ua="-" us="-" ut="-" ul="-" cs=-
94.112.41.18 - - [23/Nov/2020:20:35:17 +0100] "GET / HTTP/2.0" 200 1 "https://uhl.site/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4327.0 Safari/537.36" "-" "uhl.site" sn="uhl.site" rt=0.000 ua="-" us="-" ut="-" ul="-" cs=-
94.112.41.18 - - [23/Nov/2020:20:35:17 +0100] "GET /favicon.ico HTTP/2.0" 302 2150 "https://uhl.site/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4327.0 Safari/537.36" "-" "uhl.site" sn="uhl.site" rt=0.001 ua="-" us="-" ut="-" ul="-" cs=HIT
94.112.41.18 - - [23/Nov/2020:20:35:17 +0100] "GET /wp-content/uploads/2020/11/cropped-web-programming-32x32.png HTTP/2.0" 200 560 "https://uhl.site/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4327.0 Safari/537.36" "-" "uhl.site" sn="uhl.site" rt=0.001 ua="-" us="-" ut="-" ul="-" cs=-

And errror log:

2020/11/23 20:36:08 [debug] 3767127#0: *491 CollectAccumulatedWrites, 0
2020/11/23 20:36:21 [notice] 3767127#0: *492 "443" matches "443", client: 35.200.109.119, server: uhl.site, request: "HEAD / HTTP/1.1", host: "uhl.site"
2020/11/23 20:36:21 [notice] 3767127#0: *492 "/wp-admin/|/xmlrpc.php|wp-.*.php|/feed/|index.php|sitemap(_index)?.xml" does not match "/", client: 35.200.109.119, server: uhl.site, request: "HEAD / HTTP/1.1", host: "uhl.site"
2020/11/23 20:36:21 [notice] 3767127#0: *492 "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_no_cache|wordpress_logged_in" does not match "", client: 35.200.109.119, server: uhl.site, request: "HEAD / HTTP/1.1", host: "uhl.site"
2020/11/23 20:36:21 [notice] 3767127#0: *492 "/wp-admin$" does not match "/", client: 35.200.109.119, server: uhl.site, request: "HEAD / HTTP/1.1", host: "uhl.site"
2020/11/23 20:36:21 [info] 3767127#0: [ngx_pagespeed 1.13.35.2-0] Trying to serve rewritten resource in-place: https://uhl.site/
2020/11/23 20:36:21 [info] 3767127#0: [ngx_pagespeed 1.13.35.2-0] Cache entry is expired: https://uhl.site/ (fragment=uhl.site)
2020/11/23 20:36:21 [info] 3767127#0: [ngx_pagespeed 1.13.35.2-0] Could not rewrite resource in-place because URL is not in cache: https://uhl.site/
2020/11/23 20:36:21 [notice] 3767127#0: *492 "443" matches "443", client: 35.200.109.119, server: uhl.site, request: "HEAD / HTTP/1.1", host: "uhl.site"
2020/11/23 20:36:21 [notice] 3767127#0: *492 "/wp-admin/|/xmlrpc.php|wp-.*.php|/feed/|index.php|sitemap(_index)?.xml" does not match "/", client: 35.200.109.119, server: uhl.site, request: "HEAD / HTTP/1.1", host: "uhl.site"
2020/11/23 20:36:21 [notice] 3767127#0: *492 "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_no_cache|wordpress_logged_in" does not match "", client: 35.200.109.119, server: uhl.site, request: "HEAD / HTTP/1.1", host: "uhl.site"
2020/11/23 20:36:21 [notice] 3767127#0: *492 "/wp-admin$" does not match "/index.php", client: 35.200.109.119, server: uhl.site, request: "HEAD / HTTP/1.1", host: "uhl.site"
2020/11/23 20:36:21 [info] 3767127#0: [ngx_pagespeed 1.13.35.2-0] Trying to serve rewritten resource in-place: https://uhl.site/
2020/11/23 20:36:21 [info] 3767127#0: [ngx_pagespeed 1.13.35.2-0] Cache entry is expired: https://uhl.site/ (fragment=uhl.site)
2020/11/23 20:36:21 [info] 3767127#0: [ngx_pagespeed 1.13.35.2-0] Could not rewrite resource in-place because URL is not in cache: https://uhl.site/
2020/11/23 20:36:21 [debug] 3767127#0: *492 CollectAccumulatedWrites, 0
2020/11/23 20:36:25 [notice] 3767127#0: *493 "443" matches "443", client: 35.246.163.41, server: uhl.site, request: "HEAD / HTTP/1.1", host: "uhl.site"
2020/11/23 20:36:25 [notice] 3767127#0: *493 "/wp-admin/|/xmlrpc.php|wp-.*.php|/feed/|index.php|sitemap(_index)?.xml" does not match "/", client: 35.246.163.41, server: uhl.site, request: "HEAD / HTTP/1.1", host: "uhl.site"
2020/11/23 20:36:25 [notice] 3767127#0: *493 "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_no_cache|wordpress_logged_in" does not match "", client: 35.246.163.41, server: uhl.site, request: "HEAD / HTTP/1.1", host: "uhl.site"
2020/11/23 20:36:25 [notice] 3767127#0: *493 "/wp-admin$" does not match "/", client: 35.246.163.41, server: uhl.site, request: "HEAD / HTTP/1.1", host: "uhl.site"
2020/11/23 20:36:25 [info] 3767127#0: [ngx_pagespeed 1.13.35.2-0] Trying to serve rewritten resource in-place: https://uhl.site/
2020/11/23 20:36:25 [info] 3767127#0: [ngx_pagespeed 1.13.35.2-0] Cache entry is expired: https://uhl.site/ (fragment=uhl.site)
2020/11/23 20:36:25 [info] 3767127#0: [ngx_pagespeed 1.13.35.2-0] Could not rewrite resource in-place because URL is not in cache: https://uhl.site/
2020/11/23 20:36:25 [notice] 3767127#0: *493 "443" matches "443", client: 35.246.163.41, server: uhl.site, request: "HEAD / HTTP/1.1", host: "uhl.site"
2020/11/23 20:36:25 [notice] 3767127#0: *493 "/wp-admin/|/xmlrpc.php|wp-.*.php|/feed/|index.php|sitemap(_index)?.xml" does not match "/", client: 35.246.163.41, server: uhl.site, request: "HEAD / HTTP/1.1", host: "uhl.site"
2020/11/23 20:36:25 [notice] 3767127#0: *493 "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_no_cache|wordpress_logged_in" does not match "", client: 35.246.163.41, server: uhl.site, request: "HEAD / HTTP/1.1", host: "uhl.site"
2020/11/23 20:36:25 [notice] 3767127#0: *493 "/wp-admin$" does not match "/index.php", client: 35.246.163.41, server: uhl.site, request: "HEAD / HTTP/1.1", host: "uhl.site"
2020/11/23 20:36:25 [info] 3767127#0: [ngx_pagespeed 1.13.35.2-0] Trying to serve rewritten resource in-place: https://uhl.site/
2020/11/23 20:36:25 [info] 3767127#0: [ngx_pagespeed 1.13.35.2-0] Cache entry is expired: https://uhl.site/ (fragment=uhl.site)
2020/11/23 20:36:25 [info] 3767127#0: [ngx_pagespeed 1.13.35.2-0] Could not rewrite resource in-place because URL is not in cache: https://uhl.site/
2020/11/23 20:36:25 [debug] 3767127#0: *493 CollectAccumulatedWrites, 0
uhlhosting commented 3 years ago

Isnt possible that nginx cache is conflicting with pagespeed?

Lofesa commented 3 years ago

Is possibel, Ican´t use nginx cache....

I can´t get any 404 or even 501 only ERR_CONNECTION_REFUSED message.

These are with the config i have send?

uhlhosting commented 3 years ago

@Lofesa i can't use that config in current environment without breaking one of them. If you refresh the page several times, you will end up seeing the page load. Like I said are several statuses that spin between them. btw at the time you tested, i am recompilling nginx now. il inform when im ready to test.

uhlhosting commented 3 years ago

Managed to recompile it, tweaked a bit more and removed what is not needed. Still gettings same results, I also disabled FastCGI cache. Redis cache in wp side, and left pagespeed + redis . I heard there are issues with gzip on? Should I disable gzip also, same for brotli?

uhlhosting commented 3 years ago

Latest console logs:

[Tue, 24 Nov 2020 01:06:40 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:06:40 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:06:35 GMT] [Info] [3868463] Unable to insert object of size: 38080, cache limit is: 35616
[Tue, 24 Nov 2020 01:06:19 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:06:19 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:06:19 GMT] [Info] [3874643] HTTPCache key=https://uhl.site/ fragment=uhl.site: remembering recent failure for 299 seconds.
[Tue, 24 Nov 2020 01:06:19 GMT] [Info] [3874643] HTTPCache key=https://uhl.site/ fragment=uhl.site: remembering recent failure for 299 seconds.
[Tue, 24 Nov 2020 01:06:19 GMT] [Error] [3868463] PageSpeed Serf fetch failure rate extremely high; only 0 of 534 recent fetches fully successful; is fetching working?
[Tue, 24 Nov 2020 01:06:19 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:06:19 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:06:19 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:06:19 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:06:19 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:06:19 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:06:19 GMT] [Info] [3874643] Cache entry is expired: https://uhl.site/ (fragment=uhl.site)
[Tue, 24 Nov 2020 01:06:19 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:06:19 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:06:19 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:06:19 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:06:19 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:06:19 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:06:19 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:06:19 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:06:19 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:06:19 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:06:19 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:06:19 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:06:19 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:06:19 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:06:19 GMT] [Info] [3868463] Unable to insert object of size: 38080, cache limit is: 35616
[Tue, 24 Nov 2020 01:02:20 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:02:20 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:02:20 GMT] [Info] [3874643] https://uhl.site/asset-manifest.json:37: CSS parser increased size of CSS file https://uhl.site/asset-manifest.json by 17 bytes.
[Tue, 24 Nov 2020 01:02:20 GMT] [Info] [3874643] HTTPCache key=https://uhl.site/asset-manifest.json fragment=uhl.site: remembering recent failure for 299 seconds.
[Tue, 24 Nov 2020 01:02:20 GMT] [Info] [3874643] HTTPCache key=https://uhl.site/asset-manifest.json fragment=uhl.site: remembering recent failure for 299 seconds.
[Tue, 24 Nov 2020 01:02:20 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:02:20 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:02:20 GMT] [Info] [3874643] https://uhl.site/wp-content/uploads/2020/11/xavatar,402x.png.pagespeed.ic.%E2%80%A6:37: CSS parser increased size of CSS file https://uhl.site/wp-content/uploads/2020/11/xavatar,402x.png.pagespeed.ic.%E2%80%A6 by 37 bytes.
[Tue, 24 Nov 2020 01:02:20 GMT] [Info] [3874643] HTTPCache key=https://uhl.site/wp-content/uploads/2020/11/xavatar,402x.png.pagespeed.ic.%E2%80%A6 fragment=uhl.site: remembering recent failure for 299 seconds.
[Tue, 24 Nov 2020 01:02:20 GMT] [Info] [3874643] HTTPCache key=https://uhl.site/wp-content/uploads/2020/11/xavatar,402x.png.pagespeed.ic.%E2%80%A6 fragment=uhl.site: remembering recent failure for 299 seconds.
[Tue, 24 Nov 2020 01:02:20 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:02:20 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:02:20 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:02:20 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:02:20 GMT] [Info] [3874643] https://uhl.site/wp-content/uploads/2020/11/xl-2,402x.png.pagespeed.ic.XxM%E2%80%A6:37: CSS parser increased size of CSS file https://uhl.site/wp-content/uploads/2020/11/xl-2,402x.png.pagespeed.ic.XxM%E2%80%A6 by 37 bytes.
[Tue, 24 Nov 2020 01:02:20 GMT] [Info] [3874643] HTTPCache key=https://uhl.site/wp-content/uploads/2020/11/xl-2,402x.png.pagespeed.ic.XxM%E2%80%A6 fragment=uhl.site: remembering recent failure for 299 seconds.
[Tue, 24 Nov 2020 01:02:20 GMT] [Info] [3874643] HTTPCache key=https://uhl.site/wp-content/uploads/2020/11/xl-2,402x.png.pagespeed.ic.XxM%E2%80%A6 fragment=uhl.site: remembering recent failure for 299 seconds.
[Tue, 24 Nov 2020 01:02:20 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:02:20 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:02:20 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:02:20 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:02:20 GMT] [Info] [3874643] https://uhl.site/wp-content/uploads/2020/11/xi-5,402x.jpg.pagespeed.ic.8bf%E2%80%A6:37: CSS parser increased size of CSS file https://uhl.site/wp-content/uploads/2020/11/xi-5,402x.jpg.pagespeed.ic.8bf%E2%80%A6 by 37 bytes.
[Tue, 24 Nov 2020 01:02:20 GMT] [Info] [3874643] HTTPCache key=https://uhl.site/wp-content/uploads/2020/11/xi-5,402x.jpg.pagespeed.ic.8bf%E2%80%A6 fragment=uhl.site: remembering recent failure for 299 seconds.
[Tue, 24 Nov 2020 01:02:20 GMT] [Info] [3874643] HTTPCache key=https://uhl.site/wp-content/uploads/2020/11/xi-5,402x.jpg.pagespeed.ic.8bf%E2%80%A6 fragment=uhl.site: remembering recent failure for 299 seconds.
[Tue, 24 Nov 2020 01:02:20 GMT] [Info] [3874643] https://uhl.site/wp-content/uploads/2020/11/xi-4,402x.jpg.pagespeed.ic.hzZ%E2%80%A6:37: CSS parser increased size of CSS file https://uhl.site/wp-content/uploads/2020/11/xi-4,402x.jpg.pagespeed.ic.hzZ%E2%80%A6 by 37 bytes.
[Tue, 24 Nov 2020 01:02:20 GMT] [Info] [3874643] HTTPCache key=https://uhl.site/wp-content/uploads/2020/11/xi-4,402x.jpg.pagespeed.ic.hzZ%E2%80%A6 fragment=uhl.site: remembering recent failure for 299 seconds.
[Tue, 24 Nov 2020 01:02:20 GMT] [Info] [3874643] HTTPCache key=https://uhl.site/wp-content/uploads/2020/11/xi-4,402x.jpg.pagespeed.ic.hzZ%E2%80%A6 fragment=uhl.site: remembering recent failure for 299 seconds.
[Tue, 24 Nov 2020 01:02:20 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:02:20 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:02:20 GMT] [Info] [3874643] https://uhl.site/wp-content/uploads/2020/11/xi-3,402x.jpg.pagespeed.ic.Lon%E2%80%A6:37: CSS parser increased size of CSS file https://uhl.site/wp-content/uploads/2020/11/xi-3,402x.jpg.pagespeed.ic.Lon%E2%80%A6 by 37 bytes.
[Tue, 24 Nov 2020 01:02:20 GMT] [Info] [3874643] HTTPCache key=https://uhl.site/wp-content/uploads/2020/11/xi-3,402x.jpg.pagespeed.ic.Lon%E2%80%A6 fragment=uhl.site: remembering recent failure for 299 seconds.
[Tue, 24 Nov 2020 01:02:20 GMT] [Info] [3874643] HTTPCache key=https://uhl.site/wp-content/uploads/2020/11/xi-3,402x.jpg.pagespeed.ic.Lon%E2%80%A6 fragment=uhl.site: remembering recent failure for 299 seconds.
[Tue, 24 Nov 2020 01:02:15 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:02:15 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:02:15 GMT] [Info] [3874643] https://uhl.site/wp-content/uploads/2020/11/xi-1,402x.jpg.pagespeed.ic.V32%E2%80%A6:37: CSS parser increased size of CSS file https://uhl.site/wp-content/uploads/2020/11/xi-1,402x.jpg.pagespeed.ic.V32%E2%80%A6 by 37 bytes.
[Tue, 24 Nov 2020 01:02:15 GMT] [Info] [3874643] HTTPCache key=https://uhl.site/wp-content/uploads/2020/11/xi-1,402x.jpg.pagespeed.ic.V32%E2%80%A6 fragment=uhl.site: remembering recent failure for 299 seconds.
[Tue, 24 Nov 2020 01:02:15 GMT] [Info] [3874643] HTTPCache key=https://uhl.site/wp-content/uploads/2020/11/xi-1,402x.jpg.pagespeed.ic.V32%E2%80%A6 fragment=uhl.site: remembering recent failure for 299 seconds.
[Tue, 24 Nov 2020 01:02:15 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:02:15 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:02:11 GMT] [Info] [3874643] Lone background-position found: Cannot sprite.
[Tue, 24 Nov 2020 01:02:11 GMT] [Info] [3874643] No permission to rewrite 'https://fancyfreelancer.oxy.host/wp-content/uploads/sites/58/2019/02/pexels-photo-529926.jpeg'
[Tue, 24 Nov 2020 01:02:10 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:02:10 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:02:10 GMT] [Info] [3874643] https://uhl.site/wp-content/uploads/2020/11/xi-2,402x-1.jpg.pagespeed.ic.i%E2%80%A6:37: CSS parser increased size of CSS file https://uhl.site/wp-content/uploads/2020/11/xi-2,402x-1.jpg.pagespeed.ic.i%E2%80%A6 by 37 bytes.
[Tue, 24 Nov 2020 01:02:10 GMT] [Info] [3874643] HTTPCache key=https://uhl.site/wp-content/uploads/2020/11/xi-2,402x-1.jpg.pagespeed.ic.i%E2%80%A6 fragment=uhl.site: remembering recent failure for 299 seconds.
[Tue, 24 Nov 2020 01:02:10 GMT] [Info] [3874643] HTTPCache key=https://uhl.site/wp-content/uploads/2020/11/xi-2,402x-1.jpg.pagespeed.ic.i%E2%80%A6 fragment=uhl.site: remembering recent failure for 299 seconds.
[Tue, 24 Nov 2020 01:02:10 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:02:10 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:02:10 GMT] [Info] [3874643] https://uhl.site/wp-content/uploads/2020/11/xi-1,402x-1.jpg.pagespeed.ic.Y%E2%80%A6:37: CSS parser increased size of CSS file https://uhl.site/wp-content/uploads/2020/11/xi-1,402x-1.jpg.pagespeed.ic.Y%E2%80%A6 by 37 bytes.
[Tue, 24 Nov 2020 01:02:10 GMT] [Info] [3874643] HTTPCache key=https://uhl.site/wp-content/uploads/2020/11/xi-1,402x-1.jpg.pagespeed.ic.Y%E2%80%A6 fragment=uhl.site: remembering recent failure for 299 seconds.
[Tue, 24 Nov 2020 01:02:10 GMT] [Info] [3874643] HTTPCache key=https://uhl.site/wp-content/uploads/2020/11/xi-1,402x-1.jpg.pagespeed.ic.Y%E2%80%A6 fragment=uhl.site: remembering recent failure for 299 seconds.
[Tue, 24 Nov 2020 01:02:10 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:02:10 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:02:10 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:02:10 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:02:10 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:02:10 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:02:10 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:02:10 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:02:10 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:02:10 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:02:10 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:02:10 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:02:10 GMT] [Info] [3874643] Lone background-position found: Cannot sprite.
[Tue, 24 Nov 2020 01:02:10 GMT] [Info] [3874643] No permission to rewrite 'https://fancyfreelancer.oxy.host/wp-content/uploads/sites/58/2019/02/pexels-photo-529926.jpeg'
[Tue, 24 Nov 2020 01:02:07 GMT] [Info] [3868463] Unable to insert object of size: 38080, cache limit is: 35616
[Tue, 24 Nov 2020 01:02:07 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:02:07 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:00:50 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:00:50 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:00:50 GMT] [Info] [3874643] Inside CSS: https://uhl.site/wp-content/uploads/oxygen/css/universal.css?cache=1606179525&ver=5.5.3: Image MIME type could not be discovered from reading magic bytes; rewriting dropped.
[Tue, 24 Nov 2020 01:00:50 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:00:50 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:00:50 GMT] [Info] [3874643] HTTPCache key=https://uhl.site/ fragment=uhl.site: remembering recent failure for 177 seconds.
[Tue, 24 Nov 2020 01:00:50 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:00:50 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:00:50 GMT] [Info] [3874643] HTTPCache key=https://uhl.site/ fragment=uhl.site: remembering recent failure for 177 seconds.
[Tue, 24 Nov 2020 01:00:50 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:00:50 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:00:50 GMT] [Info] [3874643] Lone background-position found: Cannot sprite.
[Tue, 24 Nov 2020 01:00:50 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:00:50 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:00:50 GMT] [Info] [3868463] Need to check cache size against target 1048576000
[Tue, 24 Nov 2020 01:00:50 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
[Tue, 24 Nov 2020 01:00:50 GMT] [Info] [3868463] Unable to insert object of size: 38080, cache limit is: 35616
[Tue, 24 Nov 2020 00:58:48 GMT] [Info] [3874643] Inside CSS: https://uhl.site/wp-content/uploads/oxygen/css/universal.css?cache=1606179525&ver=5.5.3: Image MIME type could not be discovered from reading magic bytes; rewriting dropped.
[Tue, 24 Nov 2020 00:58:48 GMT] [Info] [3874643] HTTPCache key=https://uhl.site/ fragment=uhl.site: remembering recent failure for 299 seconds.
[Tue, 24 Nov 2020 00:58:48 GMT] [Info] [3874643] HTTPCache key=https://uhl.site/ fragment=uhl.site: remembering recent failure for 299 seconds.
[Tue, 24 Nov 2020 00:58:48 GMT] [Info] [3868463] Unable to insert object of size: 38080, cache limit is: 35616
[Tue, 24 Nov 2020 00:58:48 GMT] [Info] [3874643] HTTPCache key=https://uhl.site/ fragment=uhl.site: remembering recent failure for 299 seconds.
[Tue, 24 Nov 2020 00:58:48 GMT] [Info] [3874643] HTTPCache key=https://uhl.site/ fragment=uhl.site: remembering recent failure for 299 seconds.
[Tue, 24 Nov 2020 00:58:48 GMT] [Info] [3874643] Inside CSS: https://uhl.site/wp-content/uploads/oxygen/css/universal.css?cache=1606179525&ver=5.5.3: Image MIME type could not be discovered from reading magic bytes; rewriting dropped.
[Tue, 24 Nov 2020 00:58:48 GMT] [Info] [3874643] Cache entry is expired: https://uhl.site/ (fragment=uhl.site)
[Tue, 24 Nov 2020 00:52:46 GMT] [Info] [3874643] HTTPCache key=https://137.74.247.207/ fragment=137.74.247.207: remembering recent failure for 299 seconds.
[Tue, 24 Nov 2020 00:52:46 GMT] [Info] [3874643] HTTPCache key=https://137.74.247.207/ fragment=137.74.247.207: remembering recent failure for 299 seconds.
[Tue, 24 Nov 2020 00:52:27 GMT] [Info] [3874643] https://uhl.site/?action=ct_render_innercontent&nonce=4e7b57bd83&post_id=125:115: CSS parser increased size of CSS file https://uhl.site/?action=ct_render_innercontent&nonce=4e7b57bd83&post_id=125 by 57 bytes.
[Tue, 24 Nov 2020 00:52:27 GMT] [Info] [3874643] HTTPCache key=https://uhl.site/?action=ct_render_innercontent&nonce=4e7b57bd83&post_id=125 fragment=uhl.site: remembering recent failure for 299 seconds.
[Tue, 24 Nov 2020 00:52:27 GMT] [Info] [3874643] HTTPCache key=https://uhl.site/?action=ct_render_innercontent&nonce=4e7b57bd83&post_id=125 fragment=uhl.site: remembering recent failure for 299 seconds.
[Tue, 24 Nov 2020 00:52:27 GMT] [Info] [3874643] Lone background-position found: Cannot sprite.

Nginx main debug log:

2020/11/24 02:13:52 [warn] 3874643#0: [ngx_pagespeed 1.13.35.2-0] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
2020/11/24 02:13:52 [info] 3874643#0: [ngx_pagespeed 1.13.35.2-0] Need to check cache size against target 1048576000
2020/11/24 02:13:53 [debug] 3874643#0: [ngx_pagespeed 1.13.35.2-0] [1124/021353:VERBOSE1:proxy_fetch.cc(532)] Attaching RewriteDriver 0x7fd621ccb000 to HtmlRewriter 0x7fd63b301c00
2020/11/24 02:13:53 [debug] 3874643#0: [ngx_pagespeed 1.13.35.2-0] [1124/021353:VERBOSE1:proxy_fetch.cc(577)] Parse successfully started.
2020/11/24 02:13:53 [debug] 3874643#0: [ngx_pagespeed 1.13.35.2-0] [1124/021353:VERBOSE1:proxy_fetch.cc(993)] Fetch result:1 https://uhl.site/ : 200
2020/11/24 02:13:53 [debug] 3874643#0: [ngx_pagespeed 1.13.35.2-0] [1124/021353:VERBOSE1:proxy_fetch.cc(532)] Attaching RewriteDriver 0x7fd621ccb000 to HtmlRewriter 0x7fd61224fe00
2020/11/24 02:13:53 [debug] 3874643#0: [ngx_pagespeed 1.13.35.2-0] [1124/021353:VERBOSE1:proxy_fetch.cc(577)] Parse successfully started.
2020/11/24 02:13:53 [debug] 3874643#0: [ngx_pagespeed 1.13.35.2-0] [1124/021353:VERBOSE1:proxy_fetch.cc(993)] Fetch result:1 https://uhl.site/ : 200
2020/11/24 02:13:53 [info] 3874643#0: [ngx_pagespeed 1.13.35.2-0] Unable to insert object of size: 38080, cache limit is: 35616
2020/11/24 02:13:56 [warn] 3874643#0: [ngx_pagespeed 1.13.35.2-0] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
2020/11/24 02:13:56 [info] 3874643#0: [ngx_pagespeed 1.13.35.2-0] Need to check cache size against target 1048576000
2020/11/24 02:13:56 [warn] 3874643#0: [ngx_pagespeed 1.13.35.2-0] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
2020/11/24 02:13:56 [info] 3874643#0: [ngx_pagespeed 1.13.35.2-0] Need to check cache size against target 1048576000
2020/11/24 02:13:56 [warn] 3874643#0: [ngx_pagespeed 1.13.35.2-0] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
2020/11/24 02:13:56 [info] 3874643#0: [ngx_pagespeed 1.13.35.2-0] Need to check cache size against target 1048576000
2020/11/24 02:13:56 [warn] 3874643#0: [ngx_pagespeed 1.13.35.2-0] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
2020/11/24 02:13:56 [info] 3874643#0: [ngx_pagespeed 1.13.35.2-0] Need to check cache size against target 1048576000
2020/11/24 02:14:04 [debug] 3874643#0: [ngx_pagespeed 1.13.35.2-0] [1124/021404:VERBOSE1:proxy_fetch.cc(532)] Attaching RewriteDriver 0x7fd63b1cd800 to HtmlRewriter 0x7fd63b301d00
2020/11/24 02:14:04 [debug] 3874643#0: [ngx_pagespeed 1.13.35.2-0] [1124/021404:VERBOSE1:proxy_fetch.cc(993)] Fetch result:1 https://uhl.site/ : 200
2020/11/24 02:14:05 [debug] 3874643#0: *2162 Passing on content handling for non-pagespeed resource 'http://127.0.0.1/nginx_status'
2020/11/24 02:14:09 [debug] 3874643#0: [ngx_pagespeed 1.13.35.2-0] [1124/021409:VERBOSE1:proxy_fetch.cc(532)] Attaching RewriteDriver 0x7fd63b1cd800 to HtmlRewriter 0x7fd63b301d00
2020/11/24 02:14:09 [debug] 3874643#0: [ngx_pagespeed 1.13.35.2-0] [1124/021409:VERBOSE1:proxy_fetch.cc(993)] Fetch result:1 https://uhl.site/ : 200
2020/11/24 02:14:16 [debug] 3874643#0: [ngx_pagespeed 1.13.35.2-0] [1124/021416:VERBOSE1:proxy_fetch.cc(532)] Attaching RewriteDriver 0x7fd63b1cd800 to HtmlRewriter 0x7fd63b301d00
2020/11/24 02:14:16 [debug] 3874643#0: [ngx_pagespeed 1.13.35.2-0] [1124/021416:VERBOSE1:proxy_fetch.cc(993)] Fetch result:1 https://uhl.site/ : 200
uhlhosting commented 3 years ago

Is this path normal?

/mnt/ram/pagespeed/shm_metadata_cache/snapshot/,2Fmnt/ram/pagespeed/metadata_cache

Initially I created the /mnt/ram/pagespeed/

Is this normal for pagespeed to recreate the same path?

Lofesa commented 3 years ago

Well....

[Tue, 24 Nov 2020 01:06:40 GMT] [Warning] [3868463] Failed to read cache clean timestamp /mnt/ram/pagespeed/!clean!time! Have the user that runs nginx write permission on /mnt/ram/pagespeed/? If yes, do: `touch /mnt/ram/pagespeed/!clean!time!`` logged as these user and see what happens.

[Tue, 24 Nov 2020 01:06:35 GMT] [Info] [3868463] Unable to insert object of size: 38080, cache limit is: 35616 You have some cache sort in size, not necesarly pagespeed related, maybe fastcgi cache or other

[Tue, 24 Nov 2020 01:06:19 GMT] [Info] [3874643] HTTPCache key=https://uhl.site/ fragment=uhl.site: remembering recent failure for 299 seconds. [Tue, 24 Nov 2020 01:06:19 GMT] [Error] [3868463] PageSpeed Serf fetch failure rate extremely high; only 0 of 534 recent fetches fully successful; is fetching working? Pagespeed is unable to fecht resources, and remember the failure for 299 seconds. Have you set correctly the LoadFromFile? Have you set correctly the Fechthttps?

/mnt/ram/pagespeed/shm_metadata_cache/snapshot/,2Fmnt/ram/pagespeed/metadata_cache

Seems you have enabled pagespeed ShmMetadataCacheCheckpointIntervalSec 300; This makes the metadata cache stored in shm memory to copy it to disk (in this case every 5min.) and don´t losse the info between restarts

Should I disable gzip also, same for brotli?

The problem with gzip/brotli comes when you enable gzip compression in pagespeed ( pagespeed HttpCacheCompressionLevel 9;) and will use brotli. When You use HttpCacheCompressionLevel pagespeed resources are served gziped, no matter if you have brotli in plaze. if you will all resources brotli compressed, you need set pagespeed HttpCacheCompressionLevel 0;

uhlhosting commented 3 years ago

The problem with gzip/brotli comes when you enable gzip compression in pagespeed ( pagespeed HttpCacheCompressionLevel 9;) and will use brotli. When You use HttpCacheCompressionLevel pagespeed resources are served gziped, no matter if you have brotli in plaze. if you will all resources brotli compressed, you need set pagespeed HttpCacheCompressionLevel 0;

It seems i dont have such variable set at all.

I recreated the !clean!time! and gave it rights.

I have completly disabled fastcgi cache.

Here is my last configs:

Archive 2.zip

Lofesa commented 3 years ago

At some point you have it https://gist.github.com/uhlhosting/634ce18b5f30f9dcb22e931ccc500ecc#file-nginx-conf-L104 If you use brotli and will get pagespeed resources compressed by brotli, you need to set it to 0, because the default value is 9 Why is your site downloading 2 times https//uhl.site whit diferents headers?

Captura1 Captura

uhlhosting commented 3 years ago

Hi,

I have updated the config file and since the one you had has changed, all pagespeed variables are in the files i sent in .zip.

  gzip off;
    gzip_disable "MSIE [1-6]\.";
    gzip_static on;
    gzip_comp_level 8;
    gzip_http_version 1.1;
    gzip_proxied any;
    gzip_vary on;
    gzip_buffers 16 8k;
    gzip_min_length 111;
    gzip_types *;   

  brotli off;
    brotli_static on;
    brotli_min_length 1400;
    brotli_buffers 32 8k;
    brotli_comp_level 0;
    brotli_types *;

In Safari I can notice this on console when logged in wp:

image 
text/plain
Resource Type   Image
Location
Full URL    https://uhl.site/ngx_pagespeed_beacon?ets=load:381&rload=1041&nav=0&dns=1&connect=1&req_start=197&ttfb=381&dwld=5&dom_c=675&nt=1&ifr=0&dpr=2&ref=https%3A%2F%2Fuhl.site%2Fhello-world%2F&url=https%3A%2F%2Fuhl.site%2F
Scheme  https
Host    uhl.site
Path    /ngx_pagespeed_beacon
Query String    ets=load:381&rload=1041&nav=0&dns=1&connect=1&req_start=197&ttfb=381&dwld=5&dom_c=675&nt=1&ifr=0&dpr=2&ref=https%3A%2F%2Fuhl.site%2Fhello-world%2F&url=https%3A%2F%2Fuhl.site%2F
Filename    ngx_pagespeed_beacon
Request & Response
Method

And a so called image that never seem to load properly, yet no idea what that could be, and console is not giving more.

text/html
Resource Type   Image
Location
Full URL    https://uhl.site/
Scheme  https
Host    uhl.site
Path    /
Filename    
Request & Response
Method  —
Protocol    —
Priority    —
Cached  No
Status  —
Code    —
Error   Unknown error
IP Address  —
Connection ID   —
Encoded —
Decoded —
Transferred —
Compressed  No
Compression

Also I noticed in chrome there is another error in console:

[Deprecation] chrome.loadTimes() is deprecated, instead use standardized API: Paint Timing. https://www.chromestatus.com/features/5637885046816768.
k @ (index):161
(anonymous) @ (index):163
load (async)
f.c @ (index):163
(anonymous) @ (index):165

this seems to be related with some js generated by pagespeed.

What should I try next?

I noticed that sometimes pages loads without pagespeed, and sometimes with it. Still getting the whitepage and on few refreshes its hittinng the pagespeed yet also without it.

uhlhosting commented 3 years ago

It seems now im warned by pagespeed and lighthouse that:

Enable text compression
0.16 s
Text-based resources should be served with compression (gzip, deflate or brotli) to minimize total network bytes. Learn more.

I was hoping pagespeed can run with existing compression or will serve by default compressed content to avoid such warnings.

Lofesa commented 3 years ago

Hi ngx_pagespeed_beacon are POST request that returns to the server info about the client, like the client viewport, screen size.... The POST request is needed for some pagespeed filters like the resize_rendered_image_dimensions and soo. And I have not errors with it in you page (when are not a blank page)

As far as I can see, you have excluded some files (with pagespeed Disallow) from pagespeed so these files are enterely under nginx control. When a file is under pagespeed "radar" they have an etag header like etag="W/0" or etag="PSA/xxxxxx", the files that are not conpresed have etag from nginx and don´t have content encoding header. As you can see in the images, url rewrited by pagespeed are gziped, despite you have brotli, because you don´t have set `pagespeed HttpCacheCompressionLevel 0;

This message [Deprecation] chrome.loadTimes() is deprecated, instead use standardized API: Paint Timing. comes from the filter pagespeed EnableFilters add_instrumentation; there is a PR #1802 to solve these issue, but is not applied to the pgespeed module. ` Captura

Captura1

uhlhosting commented 3 years ago

I have set the pagespeed HttpCacheCompressionLevel 0; . Still I hit the white blank pages, and another issue is that any link after uhl.site/testme.php will hit the blog page, and not a 404 page. So i think the last change suggested to change from try_files $uri =404; this try to find a file that match the uri and if not found return a 404. Better try to use this try_files $uri $uri/ /index.php$is_args$args ; this rule is already in rewrite url, since this is specific to wordpress. Yet by changing that maybe that affects now and everything I write after / with .php in end will hit the blog page, thats not expected behaviour or is this normal with pagespeed?

uhlhosting commented 3 years ago

It seems the errorrs are visible only when i am logged in, or maybe is browser specific will dig this. Also when i am logged in, I do not seem to get the blank pages at all. @Lofesa any idea what can be causing this blank pages? brotli is off, i added the mentioned variable in place, fastcgi its off. There is opcache on php, can that interfere with pagespeed in any ways?

Lofesa commented 3 years ago

Whe I have advised about this:

For example, in you fastcgi.conf have you a line like: fastcgi_param SCRIPT_FILENAME $document_root/$fastcgi_script_name; or in the uhl.site.conf there is this at line 101: try_files $uri =404; this try to find a file that match the uri and if not found return a 404. Better try to use this try_files $uri $uri/ /index.php$is_args$args ;

This file and this line are the php location for wordpress. I use some like this in a server block:

server {
        try_files $uri $uri/ /index.php$is_args$args;
        location ~ \.php(/|$) {
                fastcgi_split_path_info ^(.+?\.php)(/.*)$;
                try_files $uri =404;
                include default.d/fastphp.conf;
        }
}

What happens in your site when pagespeed is off? (this maybe achieved with https://uhl.site/?PageSpeed=off) 1.- assets have 2 cache control headers:

cache-control: max-age=31536000
cache-control: public, no-transform

2.- Are not compressed, so your config for gzip and brotli are not working, look at where you included the file with gzip and brotli config 3.- https://uhl.site is downloaded 2 times with different http headers. When pagespeed is enabled, 1 have x-page-speed header and the other not, 1 have x-cache=HIT and the other some times have x-cache= HIT and others have x-cache=From uhl.site Have you a loadbalancer in place? Why this happen? When the blank page comes the http headers don´t have x-page-speed and have x-cache=HIT Have any compress plugin in the worpress?

uhlhosting commented 3 years ago

Updated fresh config file: https://gist.github.com/uhlhosting/f2ed6e353dd04c94f0997b8b31c175b2

  1. I have reenabled gzip and brotli, here I am confused, should they be enabled or disabled. Is there a fallback mechanism or a reason pagespeed wont work on some cases, and then will revert to current config without pagespeed.

  2. Now the cache control is solved no more duplicates, it was that some files had the directive in their location, i cleaned that.

  3. this should be fixed now. not downloading 2 headers.

It seems the uhl.site document is called 2 times, one time has duplicated headers, still this seem so much a bug in pagespeed or missconfiguration, while the main uhl.site is theree with normal headers, second entry it has all duplicates and a wrong chain.

image image

There is no loadbalancer in place, this was default config, and I added the fastcgi config like here: https://forum.aapanel.com/d/593-how-to-enable-fastcgi-cache-for-wordpress-with-aapanel

. This remains a dilema, now you got all my latest config files. Maybe now after i reset the vhost will be easier to debug.

uhlhosting commented 3 years ago

With pagespeed=off seems headers arent duplicated anymore, yet the request link it is again duplicated:

image image image

difference is visible in referral policy and initiator ofcourse is simillar to to the images above.

Lofesa commented 3 years ago 
2. I have reenabled gzip and brotli, here I am confused, should they be enabled or disabled. Is there a fallback mechanism or a reason pages

Enable gzip and brotli in your nginx config, then the brownser can choose what compresion get. This unconditionally, despite the use of pagespeed module or not. All your content (not images, for that you must set the mime types in gzip and in brotli) must be compressed. The pagespeed resources (these that have .pagespeed. in the url): If you don´t set HttpCacheCompressionLevel or set it whit a value other than 0, pagespeed resources are served gziped, despite you have brotli in place. It you set it to 0, then pagespeed resources can be served gziped or brotli compressed, brownser choose.

It seems the uhl.site document is called 2 times, one time has duplicated headers, still this seem so much a bug in pagespeed or missconfiguration, while the main uhl.site is theree with normal headers, second entry it has all duplicates and a wrong chain.

Misconfiguration. Even with pagespeed set to off, there are 2 downloads, so some thing in your config (or your ISP) is doing these extra request.

uhlhosting commented 3 years ago

Well i have attached the last nginx config. I dont seem to find that. Also i doubt the provider alters this. Or this would be on every page. And is not the case.

Lofesa commented 3 years ago

Can you make a zip file with ngnx.conf and all included files include enable-php-74.conf; include /www/server/panel/vhost/rewrite/uhl.site.conf; include apps/pagespeed/pagespeed.conf; As far as I can see these are the files included in nginx.conf. If there are others files included in these, put it in the zip file too.

Some thing is making these extra request.

uhlhosting commented 3 years ago

I have attached here entire nginx/conf folder since this panel uses a custom nginx deployment folder namming it differs.

Also there is included the vhost of the site running, and entire apps/pagespeed/ conntent. Basically all configs that make the site run.

Pagespeed setup: pagespeed.tar.gz

Entire nginx config conf.tar-2.gz

Website config Vhost.zip

Rewrite config uhl.site.conf.tar.gz

Looking forward to your findings.

uhlhosting commented 3 years ago

if you wish I could provide you with a temporary login to my panel if its easier to check there in place, would just need your mail, since those links are a danger in public.

Lofesa commented 3 years ago

Issues in confiles files, regarding files in the order they appears, starting in nginx.conf:

1.- File nginx.conf line 19 you don´t need any proxy config, comment this line ( include proxy.conf;)

2.- File nginx.conf lines 59-60 these cetificate files are for your host. Pagespeed don´t use these, but the default for your OS. Read https://www.modpagespeed.com/doc/https_support#configuring_ssl_certificates

3.- File nginx.conf line 116 you have include enable-php.conf; this file have include fastcgi.conf; and include pathinfo.conf;. In this last file you have if ($fastcgi_script_name ~ "^(.+?\.php)(/.+)$") { this regular expression is malformed. the " are not needed and you have a close parenthesis that is unopened. All the stuff in this file can be changed by fastcgi_split_path_info ^(.+?\.php)(/.*)$; More simple, more fast, no if involved....

4.- File uhl.site.conf (in the vhos dir.) Here you, or the apanel team, make a decission that I don´t agree: Set all traffic in 1 server block and then check all request for the port and rewrite the url. I think is better and faster having 3 separate server block, 1 for all ports www,1 for http (port 80) and unconditionally redirect to https (port 443), so:

server {
           name www.uhl.site
           redirect 301 https://uhl.site$request_uri;
}
server {
          port 80;
          name uhl.site; 
         redirect 301 https://uhl.site$request_uri;
}
server{
        port 443;
        name uhl.site;
       all the stuff here;
}

or at least change the rewrite in line 20 with a more efective redirect.

5.- File uhl.site.conf (in the vhos dir.) line 20. Can be this the origin for the 2 downloads? Try to comment and see what happens.

6.- File uhl.site.conf (in the vhos dir.) line 39. Same issue that #3 because have include pathinfo.conf;

7.- File uhl.site.conf (in the vhos dir.) line 43. Here you include the rewrite/uhl.site.conf;. here you have the try_files $uri $uri/ /index.php?$args; you need try_files $uri $uri/ /index.php$is_args$args; Only include $args if there are any $args.

8.- I see you have a pagespeed/main.cf with pagespeed directives, but don´t see it included on any other file.

Others advices:

A.- You have the include pagespeed/pagespeed.conf at the very end of the file, try to put at the start, just when all the add_headers ends.

B.- you use 2 files enable-php.conf and enable-php-74.conf with the exact same content. Why don´t use only 1? And this file must be 

location ~ [^/]\.php(/|$)
        {
                fastcgi_split_path_info ^(.+?\.php)(/.*)$;
                try_files $uri =404;
                fastcgi_index index.php;
                include fastcgi.conf;
                fastcgi_pass  unix:/tmp/php-cgi-74.sock;
       }

I may have missed something....

You must read this for some questions : https://www.nginx.com/resources/wiki/start/topics/tutorials/config_pitfalls/

Lofesa commented 3 years ago

Have you some type of server push in place? try to disable it and see what happens with the 2 download times

uhlhosting commented 3 years ago

@nginx.conf file the pagespeed variables are old ones, and instead the main.conf is called, i have disabled proxy.conf, in the end the config look like this: https://gist.github.com/uhlhosting/79cf6a6a10488f59a3fcc23b5cc5a660

Updated php-fpm74.conf with following:

#   location ~ [^/]\.php(/|$)
#   {
#       try_files $uri =404;
#       fastcgi_pass  unix:/tmp/php-cgi-74.sock;
#       fastcgi_index index.php;
#       include fastcgi.conf;
#       include pathinfo.conf;
#   }
location ~ [^/]\.php(/|$)
        {
                fastcgi_split_path_info ^(.+?\.php)(/.*)$;
                try_files $uri =404;
                fastcgi_index index.php;
                include fastcgi.conf;
                fastcgi_pass  unix:/tmp/php-cgi-74.sock;
       }

Let;s see now if its better.

uhlhosting commented 3 years ago

It seems the white page of death is gone :) it toke us some time yet slowly is good we see improvements, it seems there still exists those 2 entries of uhl.site yet not with duplicated headers anymore.

uhlhosting commented 3 years ago

Now the only thing is, since using pagespeed would indeed expect some better results then previously: https://gtmetrix.com/compare/Qlx3cZJw/jhYJPNf3 when used with fastcgi and redis opcache stack. Any idea whats missing now for pagespeed to reduce tftb?

this is current pagespeed config:

Version: 14: on

Filters
ah  Add Head
ai  Add Instrumentation
ij  Canonicalize Javascript library URLs
cw  Collapse Whitespace
cc  Combine Css
jc  Combine Javascript
gp  Convert Gif to Png
jp  Convert Jpeg to Progressive
jw  Convert Jpeg To Webp
mc  Convert Meta Tags
pj  Convert Png to Jpeg
wa  Convert animated images to WebP
ws  When converting images to WebP, prefer lossless conversions
dd  Dedup Inlined Images
di  Delay Images
ea  Elide Attributes
ec  Cache Extend Css
ei  Cache Extend Images
ep  Cache Extend PDFs
es  Cache Extend Scripts
fc  Fallback Rewrite Css 
if  Flatten CSS Imports
hw  Flushes html
ci  Inline Css
ii  Inline Images
il  Inline @import to Link
ji  Inline Javascript
idp Insert DNS Prefetch
id  Insert Image Dimensions
js  Jpeg Subsampling
ll  Lazyload Images
ls  Local Storage Cache
cj  Move Css Above Scripts
cm  Move Css To Head
rj  Recompress Jpeg
rp  Recompress Png
rw  Recompress Webp
rc  Remove Comments
ri  Resize Images
rm  Resize Mobile Images
cf  Rewrite Css
jm  Rewrite External Javascript
jj  Rewrite Inline Javascript
cs  Rewrite Style Attributes
cu  Rewrite Style Attributes With Url
cp  Strip Image Color Profiles
md  Strip Image Meta Data

Options
  AdminPath (nap)                                      /pagespeed_admin
  AvoidRenamingIntrospectiveJavascript (aris)          True
  BeaconUrl (bu)                                       /ngx_pagespeed_beacon
  ConsolePath (ncp)                                    /pagespeed_console
  CssImageInlineMaxBytes (cii)                         0
  DisableRewriteOnNoTransform (drnt)                   False
  DownstreamCacheRewrittenPercentageThreshold (dcrpt)  95
  EnableCachePurge (euci)                              True
  EnableRewriting (e)                                  1
  FetchHttps (fhs)                                     enable,allow_self_signed
  FetchWithGzip (afg)                                  True
  FileCacheCleanIntervalMs (afcci)                     3600000
  FileCacheInodeLimit (afcl)                           1024000
  FileCachePath (afcp)                                 /mnt/ram/pagespeed
  FileCacheSizeKb (afc)                                1024000
  ForbidAllDisabledFilters (fadf)                      True
  GlobalAdminPath (ngap)                               /pagespeed_global_admin
  GlobalStatisticsPath (ngsp)                          /ngx_pagespeed_global_statistics
  HonorCsp (hcsp)                                      True
  HttpCacheCompressionLevel (hccl)                     0
  ImageInlineMaxBytes (ii)                             3072
  InPlaceResourceOptimization (ipro)                   False
  LazyloadImagesAfterOnload (llio)                     False
  LogDir (ald)                                         /var/log/pagespeed
  LowercaseHtmlNames (lh)                              True
  LRUCacheByteLimit (alcb)                             16384
  LRUCacheKbPerProcess (alcp)                          8192
  MaxCacheableContentLength (rcl)                      -1
  MaxCombinedCssBytes (xcc)                            -1
  MaxCombinedJsBytes (xcj)                             -1
  MaxInlinedPreviewImagesIndex (mdii)                  6
  MessagesPath (nmp)                                   /ngx_pagespeed_message
  MinImageSizeLowResolutionBytes (nislr)               3072
  ModifyCachingHeaders (mch)                           True
  NoTransformOptimizedImages (ntoi)                    True
  PreserveUrlRelativity (pur)                          True
  PurgeMethod (pm)                                     PURGE
  RedisServer (rds)                                    127.0.0.1:6379
  RespectVary (rv)                                     True
  RewriteLevel (l)                                     Core Filters
  Statistics (ase)                                     True
  StatisticsLogging (asle)                             True
  StatisticsLoggingIntervalMs (asli)                   60000
  StatisticsLoggingMaxFileSizeKb (aslfs)               1024
  StatisticsPath (nsp)                                 /ngx_pagespeed_statistics
  SupportNoScriptEnabled (snse)                        False
  XHeaderValue (xhv)                                   Powered By UHL.agency

Domain Lawyer
  https://uhl.site/ Auth

Invalidation Timestamp: Wed, 25 Nov 2020 21:28:39 GMT (1606339719094)
Lofesa commented 3 years ago

Don you need pagespeed EnableFilters local_storage_cache; or pagespeed EnableFilters add_instrumentation;? Whit worpress the main reason for big ttfb is the backend process, have you too much plugins in place? And now you can introduce some type of html page cache, maybe fastcgi_cache or others, to avoid time procesin creating the html page.

Have you checked this:

Have you some type of server push in place? try to disable it and see what happens with the 2 download times

uhlhosting commented 3 years ago

Well if you check the result in gtmetrix, i achieved 300ms tftb and it was same setup just no pagespeed and was fastcgi. we use a minimum of plugins.

I have removed the variables you mentioned above.

So now you say I can use fastcgi on top of this setup?

Yes I checked I cant seem to find any server push in place, could that be the waf doing such actions? yet even that one was disabled.

Lofesa commented 3 years ago

Yes you can use fastcgi_cache (for config example see https://easyengine.io/wordpress-nginx/tutorials/single-site/fastcgi-cache-with-purging/) But I don´t like these cache (is fast, I know) but, even with nginx helper plugin, don´t have a good control on cache, don´t delete cache entries when you add content to wordpress and such type of issue. Some other page cache types have more control on this. Some of these caches are worpress plugins and you may check some of these until found one that satifies your needs. I think the double download have influence in the bad timmings you have. https://www.webpagetest.org/result/201126_Di7Y_427849cae75afc14a24b438b45e2e0b3/

uhlhosting commented 3 years ago

Surely the double download has influence, just not sure what could cause it yet. Will try to read fastcgi and see if any changes also enable redis on wp side.

Lofesa commented 3 years ago

I still have taking a look on the issue, but can´t see the reason for this..... In an empty log file, when you hit the page, have you these 2 downloads?

About Redis... you can use it in place of memcached for spagespeed cache. Is sure and stable. You can use it too as an object cache, a intermediate cache between the wordpress and the database (https://es.wordpress.org/plugins/redis-cache/ or https://es.wordpress.org/plugins/wp-redis/) And you can use it a a page cache for wordpress ( https://es.wordpress.org/plugins/pj-page-cache-red/ )

Lofesa commented 3 years ago

Can you change to a default theme? Some like the Twenty Fifteen?

uhlhosting commented 3 years ago

i have themes disable now, since im using oxygen builder that doesnt uses a theme base.

uhlhosting commented 3 years ago

https://forum.aapanel.com/d/593-how-to-enable-fastcgi-cache-for-wordpress-with-aapanel will try again this setup, and will surely urge the aapanel team to look into suggestions made, still havent been able to figure out the duplicate, i had 300 ms with fastcgi will give it a try again.

uhlhosting commented 3 years ago

Visible difference with fastcgi on: https://gtmetrix.com/compare/OhWm2FXv/PQSzLJfz .