pagespeed with load balancer (nginx) and front-end (apache)

[salvoaranzulla]

Hi,

We have a load balancer with nginx (proxy_pass) and many front-end with apache and mod_pagespeed. We use memcached servers for mod_pagespeed cache.

I'd like to know if it's a good idea installing ngx_pagespeed on load balancer and configuring it with:

pagespeed standby; pagespeed MemcachedServers "same memcached server of mod_pagespeed of frontend";

to avoid request to Apache (and mod_pagespeed) from nginx load balancer for already optimized resources (.pagespeed. file).

Thanks, Salvatore

[jmarantz]

That is an interesting idea, and I can see the potential value. But the system isn't quite designed for that use-case.

The reason, I'm guessing, that you'd prefer this architecture to simplify using ngx_pagespeed in the proxy and removing it from the Apache instances, is that you want to keep the proxy lean, and have the Apache instances take care of all the hard work of (say) image optimization and URL fetching.

The problem in practice is that when the standby ngx_pagespeed instance sees a .pagespeed. URL that is not in the cache, it will need to serve it; it won't proxy the request through to the backend servers. To serve it, it will need to fetch it and optimize it. Maybe if your cache TTLs are long and your memcached instance is large, that will rarely happen. But in that case you will do pretty well at steady-state with just ngx_pagespeed at the proxy.

On Thu, Feb 6, 2020 at 7:48 AM salvoaranzulla notifications@github.com wrote:

Hi,

We have a lod balancer with nginx (proxy_pass) and many front-end with apache and mod_pagespeed. We use memcached servers for mod_pagespeed cache.

I'd like to know if it's a good idea installing ngx_pagespeed on load balancer and configuring it with:

pagespeed standby; pagespeed MemcachedServers "same memcached server of mod_pagespeed of frontend";

to avoid request to Apache (and mod_pagespeed) from nginx load balancer for already optimized resources (.pagespeed. file).

Thanks, Salvatore

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/apache/incubator-pagespeed-ngx/issues/1679?email_source=notifications&email_token=AAO2IPLXRJDDGGLFKNDQVMLRBQBKTA5CNFSM4KQ4JUGKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4ILQBRKA, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAO2IPPNREICSOCAWVSJDX3RBQBKTANCNFSM4KQ4JUGA .

[salvoaranzulla]

Hi,

Thanks for your reply and clarification.

So it's better to run mod_pagespeed in each our 4 front-end with Apache and Memcached.

I have another question. When we run mod_pagespeed in our 4 front-end and I go to Pagespeed Admin to purge cache, I always see a different timestamp for each front-end after "Everything before this time stamp is invalid":

• Do I need to purge cache in each 4 front-end?

• Do I need to use a shared folder (maybe in a NAS) in ModPagespeedFileCachePath instead of "/var/cache/pagespeed/"?

Thanks, Salvatore

On Thu, Feb 6, 2020 at 3:00 PM Joshua Marantz notifications@github.com wrote:

That is an interesting idea, and I can see the potential value. But the system isn't quite designed for that use-case.

The reason, I'm guessing, that you'd prefer this architecture to simplify using ngx_pagespeed in the proxy and removing it from the Apache instances, is that you want to keep the proxy lean, and have the Apache instances take care of all the hard work of (say) image optimization and URL fetching.

The problem in practice is that when the standby ngx_pagespeed instance sees a .pagespeed. URL that is not in the cache, it will need to serve it; it won't proxy the request through to the backend servers. To serve it, it will need to fetch it and optimize it. Maybe if your cache TTLs are long and your memcached instance is large, that will rarely happen. But in that case you will do pretty well at steady-state with just ngx_pagespeed at the proxy.

On Thu, Feb 6, 2020 at 7:48 AM salvoaranzulla notifications@github.com wrote:

Hi,

We have a lod balancer with nginx (proxy_pass) and many front-end with apache and mod_pagespeed. We use memcached servers for mod_pagespeed cache.

I'd like to know if it's a good idea installing ngx_pagespeed on load balancer and configuring it with:

pagespeed standby; pagespeed MemcachedServers "same memcached server of mod_pagespeed of frontend";

to avoid request to Apache (and mod_pagespeed) from nginx load balancer for already optimized resources (.pagespeed. file).

Thanks, Salvatore

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub < https://github.com/apache/incubator-pagespeed-ngx/issues/1679?email_source=notifications&email_token=AAO2IPLXRJDDGGLFKNDQVMLRBQBKTA5CNFSM4KQ4JUGKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4ILQBRKA , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AAO2IPPNREICSOCAWVSJDX3RBQBKTANCNFSM4KQ4JUGA

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/apache/incubator-pagespeed-ngx/issues/1679?email_source=notifications&email_token=ABNVCX64VZIQTEWC3MVOQP3RBQJZNA5CNFSM4KQ4JUGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEK7KA4Q#issuecomment-582918258, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNVCXYXA5PJMNN54JGVXM3RBQJZNANCNFSM4KQ4JUGA .

[Lofesa]

Hi

• Do I need to use a shared folder (maybe in a NAS) in ModPagespeedFileCachePath instead of "/var/cache/pagespeed/"?

I think is a bad idea. The time to load resources from a shared disk mayincrease the time to serve the resource

• Do I need to purge cache in each 4 front-end?

If you use 4 disk cache, yes you need to purge each. But you can manage to store all the optimized resources in memcached, in latest versions it can stored big object (version 1.4.2 or later) with the -I option, or change memcached by redis that don´t have object size limits. To do so, you need to no configure any cache, only the file cache cause if not configured pagespeed don´t workbut not the ModPagespeedCreateSharedMemoryMetadataCacheor and disable the ModPagespeedDefaultSharedMemoryCacheKB setting their size to 0. And maybe the same ModPagespeedCacheFragment some_token in all 4 servers.

In ths way all metadata and optimized resources are stored in memcached or redis I have not tried this with multile servers, but I have 1 server and can´t use the disk.

[jmarantz]

One clarification: even if you use a shared cache like memcached or redis, you need to purge the cache on each server. This is because purging the cache doesn't actually remove the item from the cache; it adds metadata local to the server about what's been purged at what date.

This should be easy to script by sending PURGE requests to each server, e.g. via curl.

On Thu, Feb 6, 2020 at 2:12 PM Lofesa notifications@github.com wrote:

Hi

• Do I need to use a shared folder (maybe in a NAS) in ModPagespeedFileCachePath instead of "/var/cache/pagespeed/"?

I think is a bad idea. The time to load resources from a shared disk mayincrease the time to serve the resource

• Do I need to purge cache in each 4 front-end? If you use 4 disk cache, yes you need to purge each. But you can manage to store all the optimized resources in memcached, in latest versions it can stored big object (version 1.4.2 or later) with the -I option, or change memcached by redis that don´t have object size limits. To do so, you need to no configure any cache, only the file cache cause if not configured pagespeed don´t workbut not the ModPagespeedCreateSharedMemoryMetadataCache or and disable the ModPagespeedDefaultSharedMemoryCacheKB setting their size to 0. And maybe the same ModPagespeedCacheFragment some_token in all 4 servers.

In ths way all metadata and optimized resources are stored in memcached or redis I have not tried this with multile servers, but I have 1 server and can´t use the disk.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/apache/incubator-pagespeed-ngx/issues/1679?email_source=notifications&email_token=AAO2IPPVXPRTJKD2GFL4SSLRBROJ7A5CNFSM4KQ4JUGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELANPLA#issuecomment-583063468, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAO2IPJNMLJWKXXA7UDXMULRBROJ7ANCNFSM4KQ4JUGA .

[salvoaranzulla]

Thanks for your clarification.

1) The standard limit for memcached object is 1 Mb, right?

I need to use the large option only if I have images with size > 1 mb, right?

2) >> This is because purging the cache doesn't actually remove the item from the cache; it adds metadata local to the server about what's been purged at what date.

Are these metadata saved in var/cache/pagespeed, right?

3) If I use LoadfromFile option, does mod_pagespeed check if the file is updated every 5 minutes, right?

Does pagespeed delete the cached (optimized) file and regenerate it again every 5 minutes? Or does pagespeed just check if the file is updated?

Thanks, Salvatore

Il giorno gio 6 feb 2020 alle 20:16 Joshua Marantz notifications@github.com ha scritto:

One clarification: even if you use a shared cache like memcached or redis, you need to purge the cache on each server. This is because purging the cache doesn't actually remove the item from the cache; it adds metadata local to the server about what's been purged at what date.

This should be easy to script by sending PURGE requests to each server, e.g. via curl.

On Thu, Feb 6, 2020 at 2:12 PM Lofesa notifications@github.com wrote:

Hi

• Do I need to use a shared folder (maybe in a NAS) in ModPagespeedFileCachePath instead of "//"?

I think is a bad idea. The time to load resources from a shared disk mayincrease the time to serve the resource

• Do I need to purge cache in each 4 front-end? If you use 4 disk cache, yes you need to purge each. But you can manage to store all the optimized resources in memcached, in latest versions it can stored big object (version 1.4.2 or later) with the -I option, or change memcached by redis that don´t have object size limits. To do so, you need to no configure any cache, only the file cache cause if not configured pagespeed don´t workbut not the ModPagespeedCreateSharedMemoryMetadataCache or and disable the ModPagespeedDefaultSharedMemoryCacheKB setting their size to 0. And maybe the same ModPagespeedCacheFragment some_token in all 4 servers.

In ths way all metadata and optimized resources are stored in memcached or redis I have not tried this with multile servers, but I have 1 server and can´t use the disk.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://github.com/apache/incubator-pagespeed-ngx/issues/1679?email_source=notifications&email_token=AAO2IPPVXPRTJKD2GFL4SSLRBROJ7A5CNFSM4KQ4JUGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELANPLA#issuecomment-583063468 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AAO2IPJNMLJWKXXA7UDXMULRBROJ7ANCNFSM4KQ4JUGA

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/apache/incubator-pagespeed-ngx/issues/1679?email_source=notifications&email_token=ABNVCX7LKTHDPJ4YICX4BOLRBROYVA5CNFSM4KQ4JUGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELAN3TY#issuecomment-583065039, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNVCX3SPC77UHBMUN325Z3RBROYVANCNFSM4KQ4JUGA .

[jmarantz]

1. I think that's right: 1MB. That was based on limitations in older memcached and you can configure that higher or off I think. No limit on redis.
2. Yes exactly, though you can place that file elsewhere; see https://www.modpagespeed.com/doc/system#purge_options for detail
3. LoadFromFile is more awesome than that :) It checks on every access if the timestamp is updated. Definitely only use LoadFromFile ona local hard disk or SSD, not on a network mount, as the stat() will kill performance.

On Thu, Feb 6, 2020 at 2:39 PM salvoaranzulla notifications@github.com wrote:

Thanks for your clarification.

1) The standard limit for memcached object is 1 Mb, right?

I need to use the large option only if I have images with size > 1 mb, right?

2) >> This is because purging the cache doesn't actually remove the item from the cache; it adds metadata local to the server about what's been purged at what date.

Are these metadata saved in var/cache/pagespeed, right?

3) If I use LoadfromFile option, does mod_pagespeed check if the file is updated every 5 minutes, right?

Does pagespeed delete the cached (optimized) file and regenerate it again every 5 minutes? Or does pagespeed just check if the file is updated?

Thanks, Salvatore

Il giorno gio 6 feb 2020 alle 20:16 Joshua Marantz < notifications@github.com> ha scritto:

One clarification: even if you use a shared cache like memcached or redis, you need to purge the cache on each server. This is because purging the cache doesn't actually remove the item from the cache; it adds metadata local to the server about what's been purged at what date.

This should be easy to script by sending PURGE requests to each server, e.g. via curl.

On Thu, Feb 6, 2020 at 2:12 PM Lofesa notifications@github.com wrote:

Hi

• Do I need to use a shared folder (maybe in a NAS) in ModPagespeedFileCachePath instead of "//"?

I think is a bad idea. The time to load resources from a shared disk mayincrease the time to serve the resource

• Do I need to purge cache in each 4 front-end? If you use 4 disk cache, yes you need to purge each. But you can manage to store all the optimized resources in memcached, in latest versions it can stored big object (version 1.4.2 or later) with the -I option, or change memcached by redis that don´t have object size limits. To do so, you need to no configure any cache, only the file cache cause if not configured pagespeed don´t workbut not the ModPagespeedCreateSharedMemoryMetadataCache or and disable the ModPagespeedDefaultSharedMemoryCacheKB setting their size to 0. And maybe the same ModPagespeedCacheFragment some_token in all 4 servers.

In ths way all metadata and optimized resources are stored in memcached or redis I have not tried this with multile servers, but I have 1 server and can´t use the disk.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub <

https://github.com/apache/incubator-pagespeed-ngx/issues/1679?email_source=notifications&email_token=AAO2IPPVXPRTJKD2GFL4SSLRBROJ7A5CNFSM4KQ4JUGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELANPLA#issuecomment-583063468

, or unsubscribe <

https://github.com/notifications/unsubscribe-auth/AAO2IPJNMLJWKXXA7UDXMULRBROJ7ANCNFSM4KQ4JUGA

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub < https://github.com/apache/incubator-pagespeed-ngx/issues/1679?email_source=notifications&email_token=ABNVCX7LKTHDPJ4YICX4BOLRBROYVA5CNFSM4KQ4JUGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELAN3TY#issuecomment-583065039 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/ABNVCX3SPC77UHBMUN325Z3RBROYVANCNFSM4KQ4JUGA

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/apache/incubator-pagespeed-ngx/issues/1679?email_source=notifications&email_token=AAO2IPNTM4T5GFADJ3ZG4TTRBRRNJA5CNFSM4KQ4JUGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELAQHOI#issuecomment-583074745, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAO2IPO5T44LGRIOE4KHHDLRBRRNJANCNFSM4KQ4JUGA .

[Lofesa]

1) The standard limit for memcached object is 1 Mb, right?

Yes, but, in newer versions, you can set the max size whit the -I (upper i) size. But is allways a fixed max size.

I need to use the large option only if I have images with size > 1 mb,

Images or other type of files, css, js. For example if you have enabled the extend cache filter for pdf files then the pdf is stored in the cache.

@jmarantz In the case all optimized resources are stored in a external cache, may the cache.purge file be stored in a shared disk and then all servers have the same metadata? As far as I can see the cache.purge file only stores a plain text unix time

[salvoaranzulla]

Perfect, thanks!

3. LoadFromFile is more awesome than that :) It checks on every access if the timestamp is updated. Definitely only use LoadFromFile ona local hard disk or SSD, not on a network mount, as the stat() will kill performance.

My last question is about LoadFromFile. We use a network mount for our 4 front-end.

So it’s better downloading images from localhost (Apache that use the network mount) instead of getting them with LoadFromFile, right?

Does it exists a way to use LoadFromFile with a network mount without problems? So the ModPagespeedLoadFromFileCacheTtlMs is not so useful.

If we set an Expire in 1 year in Apache, does pagespeed update the images one time at year?

We have a large memcached server (30 gb) and we want to try to cache all things.

Thanks for you help.

Salvatore

Il giorno gio 6 feb 2020 alle 20:45 Joshua Marantz notifications@github.com ha scritto:

1. I think that's right: 1MB. That was based on limitations in older memcached and you can configure that higher or off I think. No limit on redis.
2. Yes exactly, though you can place that file elsewhere; see https://www.modpagespeed.com/doc/system#purge_options for detail
3. LoadFromFile is more awesome than that :) It checks on every access if the timestamp is updated. Definitely only use LoadFromFile ona local hard disk or SSD, not on a network mount, as the stat() will kill performance.

On Thu, Feb 6, 2020 at 2:39 PM salvoaranzulla notifications@github.com wrote:

Thanks for your clarification.

1) The standard limit for memcached object is 1 Mb, right?

I need to use the large option only if I have images with size > 1 mb, right?

2) >> This is because purging the cache doesn't actually remove the item from the cache; it adds metadata local to the server about what's been purged at what date.

Are these metadata saved in var/cache/pagespeed, right?

3) If I use LoadfromFile option, does mod_pagespeed check if the file is updated every 5 minutes, right?

Does pagespeed delete the cached (optimized) file and regenerate it again every 5 minutes? Or does pagespeed just check if the file is updated?

Thanks, Salvatore

Il giorno gio 6 feb 2020 alle 20:16 Joshua Marantz < notifications@github.com> ha scritto:

One clarification: even if you use a shared cache like memcached or redis, you need to purge the cache on each server. This is because purging the cache doesn't actually remove the item from the cache; it adds metadata local to the server about what's been purged at what date.

This should be easy to script by sending PURGE requests to each server, e.g. via curl.

On Thu, Feb 6, 2020 at 2:12 PM Lofesa notifications@github.com wrote:

Hi

• Do I need to use a shared folder (maybe in a NAS) in ModPagespeedFileCachePath instead of "//"?

I think is a bad idea. The time to load resources from a shared disk mayincrease the time to serve the resource

• Do I need to purge cache in each 4 front-end? If you use 4 disk cache, yes you need to purge each. But you can manage to store all the optimized resources in memcached, in latest versions it can stored big object (version 1.4.2 or later) with the -I option, or change memcached by redis that don´t have object size limits. To do so, you need to no configure any cache, only the file cache cause if not configured pagespeed don´t workbut not the ModPagespeedCreateSharedMemoryMetadataCache or and disable the ModPagespeedDefaultSharedMemoryCacheKB setting their size to 0. And maybe the same ModPagespeedCacheFragment some_token in all 4 servers.

In ths way all metadata and optimized resources are stored in memcached or redis I have not tried this with multile servers, but I have 1 server and can´t use the disk.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub <

https://github.com/apache/incubator-pagespeed-ngx/issues/1679?email_source=notifications&email_token=AAO2IPPVXPRTJKD2GFL4SSLRBROJ7A5CNFSM4KQ4JUGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELANPLA#issuecomment-583063468

, or unsubscribe <

https://github.com/notifications/unsubscribe-auth/AAO2IPJNMLJWKXXA7UDXMULRBROJ7ANCNFSM4KQ4JUGA

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <

https://github.com/apache/incubator-pagespeed-ngx/issues/1679?email_source=notifications&email_token=ABNVCX7LKTHDPJ4YICX4BOLRBROYVA5CNFSM4KQ4JUGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELAN3TY#issuecomment-583065039

, or unsubscribe <

https://github.com/notifications/unsubscribe-auth/ABNVCX3SPC77UHBMUN325Z3RBROYVANCNFSM4KQ4JUGA

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://github.com/apache/incubator-pagespeed-ngx/issues/1679?email_source=notifications&email_token=AAO2IPNTM4T5GFADJ3ZG4TTRBRRNJA5CNFSM4KQ4JUGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELAQHOI#issuecomment-583074745 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AAO2IPO5T44LGRIOE4KHHDLRBRRNJANCNFSM4KQ4JUGA

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/apache/incubator-pagespeed-ngx/issues/1679?email_source=notifications&email_token=ABNVCX5SEOLMGYRZT6ORZHTRBRSFTA5CNFSM4KQ4JUGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELAQ34A#issuecomment-583077360, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNVCX3VVFFJUOEIIWVGMJ3RBRSFTANCNFSM4KQ4JUGA .

[salvoaranzulla]

@Lofesa I have a question about this lines:

_>> To do so, you need to no configure any cache, only the file cache cause if not configured pagespeed don´t workbut not the ModPagespeedCreateSharedMemoryMetadataCache or and disable the ModPagespeedDefaultSharedMemoryCacheKB setting their size to 0. And maybe the same ModPagespeedCacheFragment sometoken in all 4 servers.

Why do you need to disable SharedMemoryMetadataCache if you have multiple server running the same site with memcached cache?

Does it saved on memcached cache server?

[salvoaranzulla]

@Lofesa I have understand right now the reason to put

ModPagespeedDefaultSharedMemoryCacheKB 0

For the same reason, do you suggest to me to not use LRUCache, right?

[Lofesa]

@salvoaranzulla Maybe you can use it, as far as the LRUcache is a cache not shared even between the server processes. If you have configured nginx to run, say, 4 processes the you have 4 LRUcache. SharedMemory stores metadata about the optimized resurces, if you configure it in each server, one server don´t know what metadata has created an other server. If you store it in a centraliced repository, aka external cache, these metadata can be shared too.

[salvoaranzulla]

@Lofesa Thanks for your help and your patience.

I have written this configuration:

`# ----------------------------------------------------------------------

| Mod PageSpeed Configuration |

----------------------------------------------------------------------

ModPagespeed on AddOutputFilterByType MOD_PAGESPEED_OUTPUT_FILTER text/html ModPagespeedXHeaderValue "pagespeed" ModPagespeedMemcachedServers "x.x.x.x,x.x.x.x.x,x.x.x.x,x.x.x.x" # 4 memcached server ModPagespeedFileCachePath "/var/log/pagespeed" # move to tmpfs # ModPagespeedFileCacheSizeKb 102400 # enable with size of tmpfs disk ModPagespeedDefaultSharedMemoryCacheKB 0 ModPagespeedDomain "https://www.example.it" ModPagespeedMapOriginDomain "http://localhost:8989" "https://www.example.it" ModPagespeedRespectXForwardedProto on ModPagespeedEnableFilters collapse_whitespace ModPagespeedEnableFilters lazyload_images ModPagespeedCssInlineMaxBytes 55000 ModPagespeedCssImageInlineMaxBytes 5500 ModPagespeedDisallow "*/android-chrome-*.png" ModPagespeedDisallow "*/apple-touch-icon-*.png" ModPagespeedDisallow "*/apple-touch-icon.png" ModPagespeedDisallow "*/mstile-*.png" ModPagespeedDisallow "*/favicon-*.png" ModPagespeedDisallow "*/favicon.ico" ModPagespeedDisallow "*/browserconfig.xml" ModPagespeedDisallow "*/manifest.json" ModPagespeedEnableCachePurge on ModPagespeedStatisticsLogging on ModPagespeedLogDir "/var/log/pagespeed" # move to tmpfs ModPagespeedCacheFragment example RewriteEngine Off SetHandler pagespeed_global_admin AuthType Basic AuthName "Restricted Area" AuthUserFile "path/auth/.digest_pw" Require valid-user

`

I run this configuration on my 4 front-end. In front of this front-ends, I have a load balancer with SSL.

Do you suggest my any edits?

Thanks, Salvatore

[jmarantz]

I lost track of the recommendation about caches and where that came from.

If you have any image or CSS optimizations, or URL-rewrirting enabled, you must have caches in mod_pagespeed to store optimized results. mod_pagespeed is designed to serve unoptimized resources initially while running optimizations in the background. The cache is where those are stored. When LoadFromFile is used, mod_pagespeed does not need to store and cache resources from the origin, but it does need to store the optimized results. Even if you don't have optimizations enabled that require a cache, I doubt mod_pagespeed would start up without one; we never tested that to my recollection.

The ideal configuration is to use the shared-memory cache as an L1, and memcached/redis as an L2. With memcached you need a file-cache fallback. And you always need a file-cache path, which is where the cache-purge information is stored so it persists across restarts.

If for some reason, shared-memory cache cannot be used, you can use an in-memory LRU as an L1, and there will be separate caches across child apache/nginx processes.

All of these can be used in multi-server environments.

Hope this helps, -Josh

On Sat, Feb 8, 2020 at 6:46 PM Lofesa notifications@github.com wrote:

@salvoaranzulla https://github.com/salvoaranzulla Maybe you can use it, as far as the LRUcache is a cache not shared even between the server processes. If you have configured nginx to run, say, 4 processes the you have 4 LRUcache. SharedMemory stores metadata about the optimized resurces, if you configure it in each server, one server don´t know what metadata has created an other server. If you store it in a centraliced repository, aka external cache, these metadata can be shared too.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/apache/incubator-pagespeed-ngx/issues/1679?email_source=notifications&email_token=AAO2IPI4P5CVFJ2USZ3AUO3RB473RA5CNFSM4KQ4JUGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELF6N6Q#issuecomment-583788282, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAO2IPMTBQ4XIFSK3WX2IQLRB473RANCNFSM4KQ4JUGA .

[salvoaranzulla]

@jmarantz Hi Josh,

Thanks. So even if I use multi-server (4 servers) with same pagespeed configuration, do you suggest o me to use a local shared-memory cache for each server?

Salvatore

[jmarantz]

Yes, best practice is to use a local shared-memory on each server, with a shared redis/memcached backend. In that scenario, the local shared-memory cache will store metadata, which tells pagespeed things like: "if you see in the HTML, and the accept headers include image/webp, then the optimized URL to map that to is foo.png.pagespeed.ic.HASH_CODE.webp". That mapping can occur local to the apache/nginx server, without having to incur a round-trip to your cache server. That's very important because it enables delivering the HTML to the browser as quickly as possible, so that the browser can initiate the requests for the images.

The mapping algorithms that PageSpeed use from origin URL to optimized URL are deterministic, as long as each server is running the same version of pagespeed with the same options. So there's no harm in having distinct L1 caches and a shared L2. The same optimization decisions will be made, and the same hash-codes will be computed.

On Sun, Feb 9, 2020 at 11:45 AM salvoaranzulla notifications@github.com wrote:

@jmarantz https://github.com/jmarantz Hi Josh,

Thanks. So even if I use multi-server (4 servers) with same pagespeed configuration, do you suggest o me to use a local shared-memory cache for each server?

Salvatore

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/apache/incubator-pagespeed-ngx/issues/1679?email_source=notifications&email_token=AAO2IPMU7E6SPGAQEMPZPS3RCAXLVA5CNFSM4KQ4JUGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELGRSGY#issuecomment-583866651, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAO2IPM2446APRCTXSEHCELRCAXLVANCNFSM4KQ4JUGA .

[salvoaranzulla]

Hi,

Thanks for the clarification.

Does Pagespeed generate a metadata for each HTML page? If HTML page changes, does pagespeed regenerate the metadata for the HTML page?

I'm asking to you this question because we use a Wordpress plugin that generate a comment

that changes every seconds.

Do you suggest to disable this comment to help pagespeed?

Thanks, Salvatore

On Sun, Feb 9, 2020 at 7:21 PM Joshua Marantz notifications@github.com wrote:

Yes, best practice is to use a local shared-memory on each server, with a shared redis/memcached backend. In that scenario, the local shared-memory cache will store metadata, which tells pagespeed things like: "if you see in the HTML, and the accept headers include image/webp, then the optimized URL to map that to is foo.png.pagespeed.ic.HASH_CODE.webp". That mapping can occur local to the apache/nginx server, without having to incur a round-trip to your cache server. That's very important because it enables delivering the HTML to the browser as quickly as possible, so that the browser can initiate the requests for the images.

The mapping algorithms that PageSpeed use from origin URL to optimized URL are deterministic, as long as each server is running the same version of pagespeed with the same options. So there's no harm in having distinct L1 caches and a shared L2. The same optimization decisions will be made, and the same hash-codes will be computed.

On Sun, Feb 9, 2020 at 11:45 AM salvoaranzulla notifications@github.com wrote:

@jmarantz https://github.com/jmarantz Hi Josh,

Thanks. So even if I use multi-server (4 servers) with same pagespeed configuration, do you suggest o me to use a local shared-memory cache for each server?

Salvatore

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub < https://github.com/apache/incubator-pagespeed-ngx/issues/1679?email_source=notifications&email_token=AAO2IPMU7E6SPGAQEMPZPS3RCAXLVA5CNFSM4KQ4JUGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELGRSGY#issuecomment-583866651 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AAO2IPM2446APRCTXSEHCELRCAXLVANCNFSM4KQ4JUGA

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/apache/incubator-pagespeed-ngx/issues/1679?email_source=notifications&email_token=ABNVCX75NNLBFPWYSHE5KVDRCBCSNA5CNFSM4KQ4JUGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELGT6SY#issuecomment-583876427, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNVCX3FJWJ4MBRMCFLF7A3RCBCSNANCNFSM4KQ4JUGA .

[jmarantz]

No, if you have two totally unrelated html pages both of which reference the same exact image for the same Accept request-header, and the same width/height context, the metadata cache entry will be the same and the optimized image will be shared too.

On Sun, Feb 9, 2020, 1:33 PM salvoaranzulla notifications@github.com wrote:

Hi,

Thanks for the clarification.

Does Pagespeed generate a metadata for each HTML page? If HTML page changes, does pagespeed regenerate the metadata for the HTML page?

I'm asking to you this question because we use a Wordpress plugin that generate a comment

that changes every seconds.

Do you suggest to disable this comment to help pagespeed?

Thanks, Salvatore

On Sun, Feb 9, 2020 at 7:21 PM Joshua Marantz notifications@github.com wrote:

Yes, best practice is to use a local shared-memory on each server, with a shared redis/memcached backend. In that scenario, the local shared-memory cache will store metadata, which tells pagespeed things like: "if you see in the HTML, and the accept headers include image/webp, then the optimized URL to map that to is foo.png.pagespeed.ic.HASH_CODE.webp". That mapping can occur local to the apache/nginx server, without having to incur a round-trip to your cache server. That's very important because it enables delivering the HTML to the browser as quickly as possible, so that the browser can initiate the requests for the images.

The mapping algorithms that PageSpeed use from origin URL to optimized URL are deterministic, as long as each server is running the same version of pagespeed with the same options. So there's no harm in having distinct L1 caches and a shared L2. The same optimization decisions will be made, and the same hash-codes will be computed.

On Sun, Feb 9, 2020 at 11:45 AM salvoaranzulla <notifications@github.com

wrote:

@jmarantz https://github.com/jmarantz Hi Josh,

Thanks. So even if I use multi-server (4 servers) with same pagespeed configuration, do you suggest o me to use a local shared-memory cache for each server?

Salvatore

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <

https://github.com/apache/incubator-pagespeed-ngx/issues/1679?email_source=notifications&email_token=AAO2IPMU7E6SPGAQEMPZPS3RCAXLVA5CNFSM4KQ4JUGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELGRSGY#issuecomment-583866651

, or unsubscribe <

https://github.com/notifications/unsubscribe-auth/AAO2IPM2446APRCTXSEHCELRCAXLVANCNFSM4KQ4JUGA

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub < https://github.com/apache/incubator-pagespeed-ngx/issues/1679?email_source=notifications&email_token=ABNVCX75NNLBFPWYSHE5KVDRCBCSNA5CNFSM4KQ4JUGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELGT6SY#issuecomment-583876427 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/ABNVCX3FJWJ4MBRMCFLF7A3RCBCSNANCNFSM4KQ4JUGA

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/apache/incubator-pagespeed-ngx/issues/1679?email_source=notifications&email_token=AAO2IPKMOP2OINVROFZPCETRCBEAHA5CNFSM4KQ4JUGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELGUIDY#issuecomment-583877647, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAO2IPLCTRKB5A3EPVVULEDRCBEAHANCNFSM4KQ4JUGA .

[salvoaranzulla]

Thanks. So does pagespeed create metadata only for the resources optimized, not for the HTML page?

Il giorno dom 9 feb 2020 alle 20:42 Joshua Marantz notifications@github.com ha scritto:

No, if you have two totally unrelated html pages both of which reference the same exact image for the same Accept request-header, and the same width/height context, the metadata cache entry will be the same and the optimized image will be shared too.

On Sun, Feb 9, 2020, 1:33 PM salvoaranzulla notifications@github.com wrote:

Hi,

Thanks for the clarification.

Does Pagespeed generate a metadata for each HTML page? If HTML page changes, does pagespeed regenerate the metadata for the HTML page?

I'm asking to you this question because we use a Wordpress plugin that generate a comment

that changes every seconds.

Do you suggest to disable this comment to help pagespeed?

Thanks, Salvatore

On Sun, Feb 9, 2020 at 7:21 PM Joshua Marantz notifications@github.com wrote:

Yes, best practice is to use a local shared-memory on each server, with a shared redis/memcached backend. In that scenario, the local shared-memory cache will store metadata, which tells pagespeed things like: "if you see in the HTML, and the accept headers include image/webp, then the optimized URL to map that to is foo.png.pagespeed.ic.HASH_CODE.webp". That mapping can occur local to the apache/nginx server, without having to incur a round-trip to your cache server. That's very important because it enables delivering the HTML to the browser as quickly as possible, so that the browser can initiate the requests for the images.

The mapping algorithms that PageSpeed use from origin URL to optimized URL are deterministic, as long as each server is running the same version of pagespeed with the same options. So there's no harm in having distinct L1 caches and a shared L2. The same optimization decisions will be made, and the same hash-codes will be computed.

On Sun, Feb 9, 2020 at 11:45 AM salvoaranzulla < notifications@github.com

wrote:

@jmarantz https://github.com/jmarantz Hi Josh,

Thanks. So even if I use multi-server (4 servers) with same pagespeed configuration, do you suggest o me to use a local shared-memory cache for each server?

Salvatore

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <

https://github.com/apache/incubator-pagespeed-ngx/issues/1679?email_source=notifications&email_token=AAO2IPMU7E6SPGAQEMPZPS3RCAXLVA5CNFSM4KQ4JUGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELGRSGY#issuecomment-583866651

, or unsubscribe <

https://github.com/notifications/unsubscribe-auth/AAO2IPM2446APRCTXSEHCELRCAXLVANCNFSM4KQ4JUGA

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <

https://github.com/apache/incubator-pagespeed-ngx/issues/1679?email_source=notifications&email_token=ABNVCX75NNLBFPWYSHE5KVDRCBCSNA5CNFSM4KQ4JUGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELGT6SY#issuecomment-583876427

, or unsubscribe <

https://github.com/notifications/unsubscribe-auth/ABNVCX3FJWJ4MBRMCFLF7A3RCBCSNANCNFSM4KQ4JUGA

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub < https://github.com/apache/incubator-pagespeed-ngx/issues/1679?email_source=notifications&email_token=AAO2IPKMOP2OINVROFZPCETRCBEAHA5CNFSM4KQ4JUGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELGUIDY#issuecomment-583877647 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AAO2IPLCTRKB5A3EPVVULEDRCBEAHANCNFSM4KQ4JUGA

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/apache/incubator-pagespeed-ngx/issues/1679?email_source=notifications&email_token=ABNVCXY7MDRC5JIPJRVD7BLRCBMEDA5CNFSM4KQ4JUGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELGV7MY#issuecomment-583884723, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNVCX6CJTZQWZ7OR6TBLJDRCBMEDANCNFSM4KQ4JUGA .

[oschaaf]

If you want to dive in to how caching works in PageSpeed, it is worth taking a look at jmarantz his blog post about that and some of the design docs.

[JialuZhang]

@salvoaranzulla

In your posted configuration,

<Location /pagespeed_admin>
    RewriteEngine Off
    SetHandler pagespeed_global_admin
    AuthType Basic
    AuthName "Restricted Area"
    AuthUserFile "path/auth/.digest_pw"
    Require valid-user
</Location>

the line "RewriteEngine Off“ is a silent misconfiguration. This means adding it to your system will not change any system behavior. "RewriteEngine Off" is introduced by Apache to explicitly disable all "RewriteRule" and "RewriteCond". That is to say, if you include multiple "RewriteRule" and "RewriteCond" parameters in your configuration, then instead of commenting them all, you can explicitly set “RewriteEngine Off” to disable all "RewriteRule".

More importantly, the default value of “RewriteEngine" is already an "off", so adding “RewriteEngine off" is quite unnecessary and it may cause confusion to users.

Since herein there is no "RewriteRule", deleting “RewriteEngine Off” would be ideal.

Related Apache source code snippet:

run_rewritemap_programs(server_rec *s , apr_pool_t *p){
if (conf->state == ENGINE_DISABLED) { // usage of "RewriteEngine"
  return APR_SUCCESS; // early return
rewritemap_program(...); // usage of "RewriteRule" 
}

[anime209]

Hi, i'm using nginx as load balancer with caching, where should i install pagespeed on? If i put pagespeed on nginx does it have to optimize on every request or serve from nginx cached?

[Lofesa]

Hi As a start point maybe you can read this: https://www.modpagespeed.com/doc/downstream-caching.html

[anime209]

@Lofesa Sir, does it required share cache like memcached or redis to work properly, i have 4 backend server and using memcached cloud on redislabs which is provided only 100MB memory, i would save it for php

[Lofesa]

Yes. Any pagespeed installation with more than 1 server, for better performance, need a shared cache. You may implement a sigle cache for each server, file based, but this increases resouce use and perhaps you need to grant some affinity for user request (all request from a user go to the same backend server).

[jmarantz]

From a functionality perspective, it is fine to have PageSpeed running on multiple servers with isolated caches. You will wind up storing each resource and re-optimizing each image on every server and storing it in every cache. This is because all the information needed to reconstruct each optimized asset is encoded in the rewritten URL.

The advantage of sharing a cache via memcached or Redis is that an image optimized on pagespeed server A can then be served from a request landing on pagespeed server B.

On Sun, Apr 11, 2021 at 5:04 AM Lofesa @.***> wrote:

Yes. Any pagespeed installation with more than 1 server, for better performance, need a shared cache. You may implement a sigle cache for each server, file based, but this increases resouce use and perhaps you need to grant some affinity for user request (all request from a user go to the same backend server).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/apache/incubator-pagespeed-ngx/issues/1679#issuecomment-817274563, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAO2IPNCHSAQ2VZCEYD7QILTIFQ27ANCNFSM4KQ4JUGA .