apache / incubator-pagespeed-mod

Apache module for rewriting web pages to reduce latency and bandwidth.
http://modpagespeed.com
Apache License 2.0
696 stars 158 forks source link

Log filling up with Errors: Failed to mkdir /var/cache/mod_pagespeed/ flush #1936

Open mrhdias opened 5 years ago

mrhdias commented 5 years ago

I am running apache 2.4 on CentOS Linux release 7.6.1810 (Core) and since a few months mod_pagespeed fills up my logs with the following Error:

[Thu Aug 01 16:09:01.280372 2019] [pagespeed:error] [pid 5551] [mod_pagespeed 1.13.35.2-0 @5551] Failed to mkdir /var/cache/mod_pagespeed/ flush /4i4fydsutmmoLeWCLjS8.outputlock: No such file or directory
[Thu Aug 01 16:09:01.280378 2019] [pagespeed:error] [pid 5551] [mod_pagespeed 1.13.35.2-0 @5551] Failed to mkdir /var/cache/mod_pagespeed/ flush /4i4fydsutmmoLeWCLjS8.outputlock: No such file or directory
[Thu Aug 01 16:09:01.280385 2019] [pagespeed:error] [pid 5551] [mod_pagespeed 1.13.35.2-0 @5551] Failed to mkdir /var/cache/mod_pagespeed/ flush /9RgXNIPPP5qKZ75vPHxP.outputlock: No such file or directory
[Thu Aug 01 16:09:01.280392 2019] [pagespeed:error] [pid 5551] [mod_pagespeed 1.13.35.2-0 @5551] Failed to mkdir /var/cache/mod_pagespeed/ flush /9RgXNIPPP5qKZ75vPHxP.outputlock: No such file or directory
[Thu Aug 01 16:09:01.280400 2019] [pagespeed:error] [pid 5551] [mod_pagespeed 1.13.35.2-0 @5551] Failed to mkdir /var/cache/mod_pagespeed/ flush /2f4uDxb9DWLCdQkt4oLC.outputlock: No such file or directory
[Thu Aug 01 16:09:01.280406 2019] [pagespeed:error] [pid 5551] [mod_pagespeed 1.13.35.2-0 @5551] Failed to mkdir /var/cache/mod_pagespeed/ flush /2f4uDxb9DWLCdQkt4oLC.outputlock: No such file or directory
[Thu Aug 01 16:09:01.280414 2019] [pagespeed:error] [pid 5551] [mod_pagespeed 1.13.35.2-0 @5551] Failed to mkdir /var/cache/mod_pagespeed/ flush /aGPz5p4xBMt_KrfvRDVq.outputlock: No such file or directory
[Thu Aug 01 16:09:01.280420 2019] [pagespeed:error] [pid 5551] [mod_pagespeed 1.13.35.2-0 @5551] Failed to mkdir /var/cache/mod_pagespeed/ flush /aGPz5p4xBMt_KrfvRDVq.outputlock: No such file or directory
[Thu Aug 01 16:09:01.280427 2019] [pagespeed:error] [pid 5551] [mod_pagespeed 1.13.35.2-0 @5551] Failed to mkdir /var/cache/mod_pagespeed/ flush /UvOD8bnjIQrpXYWrLqJs.outputlock: No such file or directory
[Thu Aug 01 16:09:01.280434 2019] [pagespeed:error] [pid 5551] [mod_pagespeed 1.13.35.2-0 @5551] Failed to mkdir /var/cache/mod_pagespeed/ flush /UvOD8bnjIQrpXYWrLqJs.outputlock: No such file or directory
[Thu Aug 01 16:09:01.280448 2019] [pagespeed:error] [pid 5551] [mod_pagespeed 1.13.35.2-0 @5551] Failed to mkdir /var/cache/mod_pagespeed/ flush /p08zUnjT4VcsB2heGnyr.outputlock: No such file or directory
[Thu Aug 01 16:09:01.280456 2019] [pagespeed:error] [pid 5551] [mod_pagespeed 1.13.35.2-0 @5551] Failed to mkdir /var/cache/mod_pagespeed/ flush /p08zUnjT4VcsB2heGnyr.outputlock: No such file or directory
[Thu Aug 01 16:09:05.943177 2019] [pagespeed:error] [pid 5756] [mod_pagespeed 1.13.35.2-0 @5756] Unable to find SHM segment [1]SharedCircularBuffer.:0 to attach to.
[Thu Aug 01 16:09:05.943288 2019] [pagespeed:error] [pid 5756] [mod_pagespeed 1.13.35.2-0 @5756] Unable to find SHM segment [1]globalstatistics to attach to.
[Thu Aug 01 16:09:05.943297 2019] [pagespeed:warn] [pid 5756] [mod_pagespeed 1.13.35.2-0 @5756] Problem during shared memory setup; statistics functionality unavailable.

Does anyone have the same problem and a solution for this issue?

jmarantz commented 5 years ago

Does the user running Apache have write-access to /var/cache_mod_pagespeed ?

If not, you might be able to give it access, or change the FileCachePath to point to a place where the apache user does have access.

mrhdias commented 5 years ago

Yes can write and there are many directories and files there.

But the error concerns a specific file or directory type ending in ".outputlock"! Failed to mkdir /var/cache/mod_pagespeed/ flush /Z2aU_3q6iV4ESimD4du5.outputlock: No such file or directory

$ ls -la /var/cache/
total 4
drwxr-xr-x.  8 root   root     94 Aug  1 15:28 .
drwxr-xr-x. 20 root   root    278 Jun 28 22:31 ..
drwx------.  5 apache apache   52 Jul 29 18:19 httpd
drwx------.  2 root   root     23 Jul 31 23:26 ldconfig
drwxr-xr-x. 32 root   root   4096 Aug  1 03:38 man
drwxr-xr-x.  4 apache apache   53 Aug  1 16:32 mod_pagespeed
drwxr-xr-x.  2 root   root      6 May 22 11:10 php-pear
drwxr-xr-x.  3 root   root     20 Jun  4 10:17 yum

$ ls -la /var/cache/mod_pagespeed/
total 4
drwxr-xr-x. 4 apache apache 53 Aug  1 16:32 .
drwxr-xr-x. 8 root   root   94 Aug  1 15:28 ..
-rw-------. 1 apache apache 13 Aug  1 16:32 !clean!time!
drwxr-xr-x. 4 apache apache 37 Aug  1 15:59 prop_page
drwxr-xr-x. 4 apache apache 48 Aug  1 15:57 v3

$ sudo semanage fcontext -l | grep /var/cache/mod_
/var/cache/mod_.*                                  all files          system_u:object_r:httpd_cache_t:s0 
/var/cache/mod_ssl(/.*)?                           all files          system_u:object_r:httpd_cache_t:s0 
/var/cache/mod_proxy(/.*)?                         all files          system_u:object_r:httpd_cache_t:s0 
/var/cache/mod_gnutls(/.*)?                        all files          system_u:object_r:httpd_cache_t:s0 
/var/cache/mod_pagespeed(/.*)?                     all files          system_u:object_r:httpd_cache_t:s0 
Lofesa commented 5 years ago

Hi Maybe the user running apache is not apache but www-data?

mrhdias commented 5 years ago

My Linux Server is CentOS and the user can write to the directory because the directory has contents, it is not empty. The error only concerns to a file type that ends in ".outputlock".

Example: [Thu Aug 01 16:09:01.280372 2019] [pagespeed:error] [pid 5551] [mod_pagespeed 1.13.35.2-0 @5551] Failed to mkdir /var/cache/mod_pagespeed/ flush /4i4fydsutmmoLeWCLjS8.outputlock: No such file or directory

jmarantz commented 5 years ago

Depending on what mods you have installed, it's possible that the apache root process runs under a different user than the apache child processes. I'm not if that's what's going on here, and I'm puzzled a bit as well by why you are seeing "no such file or directory" errors rather than "permission denied" errors.

I am also really puzzled by the spaces between the slashes and the "flush" in your log. Did you override ModPagespeedCacheFlushFilename in https://www.modpagespeed.com/doc/system#purge_options ? And if so, what does that line in your pagespeed.conf look like?

mrhdias commented 5 years ago

$ sudo setenforce 0 The problem go away if I disable SELinux, but this is not a good solution.

joglomedia commented 5 years ago

$ sudo setenforce 0 The problem go away if I disable SELinux, but this is not a good solution.

have you add user running Apache process to SELinux with granted right permission to specific roles/type which can do file operation?

benyaminl commented 4 years ago

Related to Kernel Bug

https://bugs.centos.org/view.php?id=15680

https://support.plesk.com/hc/en-us/articles/360014944693-Pagespeed-does-not-optimize-static-files-and-throws-errors-in-Apache-logs

$ sudo setenforce 0 The problem go away if I disable SELinux, but this is not a good solution.

have you add user running Apache process to SELinux with granted right permission to specific roles/type which can do file operation?

I already did. is there any spesific selinux things that need to be configured for mod_pagespeed?

bbeckford commented 2 years ago

Did anyone ever solve this issue? Think I'm going to have to ditch Pagespeed altogether on one server as I can't disable SELinux (even when I tested it turning off didn't stop the error logs).

benyaminl commented 2 years ago

Did anyone ever solve this issue? Think I'm going to have to ditch Pagespeed altogether on one server as I can't disable SELinux (even when I tested it turning off didn't stop the error logs).

last time I tried to add httpd_sys_rw_content, sometimes works sometimes not, then ditch. Dunno now.

bbeckford commented 2 years ago

Thanks for responding, think I'm just going to remove pagespeed for now then 🤷‍♂️

Lofesa commented 2 years ago

Hi Have tried this: https://www.modpagespeed.com/doc/faq#selinux_permissions

bbeckford commented 2 years ago

@Lofesa Thanks for the suggestion, just tried that and no change I'm afraid ☹️

Lofesa commented 2 years ago

Hi There is an rpm pakage, maybe you wish to take a look on it https://centos.pkgs.org/8/getpagespeed-x86_64/nginx-module-pagespeed-selinux-1.20.0.1.13.35.2-3.el8.gps.x86_64.rpm.html

benyaminl commented 2 years ago

It basically the same. Just sometimes when on RHEL the alert are raising for some reason. The team should specifically write context for this I think.

Regards, Benyamin Limanto sent from my Sony XPERIA XZ Premium

On Tue, 5 Apr 2022, 17:43 Lofesa, @.***> wrote:

Hi There is an rpm pakage, maybe you wish to take a look on it

https://centos.pkgs.org/8/getpagespeed-x86_64/nginx-module-pagespeed-selinux-1.20.0.1.13.35.2-3.el8.gps.x86_64.rpm.html

— Reply to this email directly, view it on GitHub https://github.com/apache/incubator-pagespeed-mod/issues/1936#issuecomment-1088546254, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAVP2NDZ5UR6GJADNQGM5ELVDQKNHANCNFSM4IIRYAJQ . You are receiving this because you commented.Message ID: @.***>

uhlhosting commented 1 year ago

We also got quite a few of:

[Sun Jan 29 00:51:04.987484 2023] [pagespeed:error] [pid 3670039:tid 139692132648704] [mod_pagespeed 1.13.35.2-0 @3670039] Failed to make directory /var/cache/mod_pagespeed/v3/eofs.dev/https,3A/,2Feofs.dev/wp-content/plugins/dev-kit: File exists
[Sun Jan 29 00:51:04.987519 2023] [pagespeed:error] [pid 3670039:tid 139692132648704] [mod_pagespeed 1.13.35.2-0 @3670039] Could not create directories for file /var/cache/mod_pagespeed/v3/eofs.dev/https,3A/,2Feofs.dev/wp-content/plugins/dev-kit/css/admin.css,3Fver=6.1.1,.temp
[Sun Jan 29 00:54:56.021140 2023] [pagespeed:error] [pid 3669996:tid 139691921737472] [mod_pagespeed 1.13.35.2-0 @3669996] Failed to make directory /var/cache/mod_pagespeed/v3/eofs.dev/https,3A/,2Feofs.dev/wp-content/uploads/2023: File exists
[Sun Jan 29 00:54:56.021180 2023] [pagespeed:error] [pid 3669996:tid 139691921737472] [mod_pagespeed 1.13.35.2-0 @3669996] Could not create directories for file /var/cache/mod_pagespeed/v3/eofs.dev/https,3A/,2Feofs.dev/wp-content/uploads/2023/01/logo-1-300x300.jpg,.temp

Any ideas if this is normal?

Lofesa commented 1 year ago

Hi @uhlhosting These messages say there are problems creating folders in the pagespeed cache. Mauybe there is a permission problem, because these folders/files are created from the user is running the aàche/httpd daemon, this user must have write permisions in these folders. Or maybe is a selinux issue, you need to configure it to permit this user/process to write in these folders