503 error, /tmp full with PHP sessions

DaWe35 commented 4 years ago

Hello!

Is there any option to prevent this from happening? I ran into it weekly, my /tmp inodes got filled. Removing session files is not the best solution, but would be better to lose some sessions than giving a 503 error.

hennaboy commented 4 years ago

Can you check if your php.ini is set to store sessions in /tmp This can also be set in .htaccess and .user.ini

This could be a number of things. php.ini set to store in /tmp Code set to write to /tmp

Options for you are;

use session_save_path in php.ini to another partition for more storage check code is the issue and if so either change it or write a cronjob to delete /tmp sessions Increase /tmp size partition

DaWe35 commented 4 years ago

@hennaboy Thank you. I use the default settings. Maybe it would make sense to make a cronjob when CyberPanel installed, wouldn't it?

meramsey commented 4 years ago

yeah like a default is not set in the lsphp .ini from looks this checked all my CyberPanel servers Centos and Ubuntu and only two of the oldest Centos one had any path defined when checking via php.ini value.

Check like this for version in $(ls /usr/local/lsws|grep lsphp); do echo ""; echo "PHP $version"; /usr/local/lsws/${version}/bin/php -i |grep -Ei 'session.save_handler|session.save_path'; done

and this

for version in $(ls /usr/local/lsws|grep lsphp); do echo ""; echo "PHP $version"; /usr/local/lsws/${version}/bin/php -r 'echo sys_get_temp_dir(), "\n";' ; done

You will see something like this

root@server:~# for version in $(ls /usr/local/lsws|grep lsphp); do echo ""; echo "PHP $version"; /usr/local/lsws/${version}/bin/php -i |grep -Ei 'session.save_handler|session.save_path'; done

PHP lsphp70
session.save_handler => files => files
session.save_path => no value => no value

PHP lsphp71
session.save_handler => files => files
session.save_path => no value => no value

PHP lsphp72
session.save_handler => files => files
session.save_path => no value => no value

PHP lsphp73
session.save_handler => files => files
session.save_path => no value => no value

PHP lsphp74
session.save_handler => files => files
session.save_path => no value => no value
root@server:~#

Only two Centos older CyberPanel ones out of all the servers had anything set and it was mismatched. I don't recall specifying it or updating that manually so its fascinating its set to something.

[root@wcloud:~]# for version in $(ls /usr/local/lsws|grep lsphp); do echo ""; echo "PHP $version"; /usr/local/lsws/${version}/bin/php -i |grep -Ei 'session.save_handler|session.save_path|session.name'; done

PHP lsphp7
session.name => PHPSESSID => PHPSESSID
session.save_handler => files => files
session.save_path => no value => no value

PHP lsphp70
session.name => PHPSESSID => PHPSESSID
session.save_handler => files => files
session.save_path => /var/lib/php/session => /var/lib/php/session

PHP lsphp71
session.name => PHPSESSID => PHPSESSID
session.save_handler => files => files
session.save_path => /var/lib/php/session => /var/lib/php/session

PHP lsphp72
session.name => PHPSESSID => PHPSESSID
session.save_handler => files => files
session.save_path => /var/lib/php/session => /var/lib/php/session

PHP lsphp73
session.name => PHPSESSID => PHPSESSID
session.save_handler => files => files
session.save_path => /var/lib/lsphp/session => /var/lib/lsphp/session

PHP lsphp74
session.name => PHPSESSID => PHPSESSID
session.save_handler => files => files
session.save_path => no value => no value
[root@wcloud:~]#

[root@cybermail:~]# for version in $(ls /usr/local/lsws|grep lsphp); do echo ""; echo "PHP $version"; /usr/local/lsws/${version}/bin/php -i |grep -Ei 'session.save_handler|session.save_path|session.gc_divisor|session.gc_maxlifetime|session.gc_probability'; done

PHP lsphp53
PHP Warning:  Unknown: It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected 'America/New_York' for 'EDT/-4.0/DST' instead in Unknown on line 0
session.gc_divisor => 1000 => 1000
session.gc_maxlifetime => 1440 => 1440
session.gc_probability => 1 => 1
session.save_handler => files => files
session.save_path => no value => no value

PHP lsphp54
PHP Warning:  Unknown: It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected the timezone 'UTC' for now, but please set date.timezone to select your timezone. in Unknown on line 0
session.gc_divisor => 1000 => 1000
session.gc_maxlifetime => 1440 => 1440
session.gc_probability => 1 => 1
session.save_handler => files => files
session.save_path => /var/lib/php/session => /var/lib/php/session

PHP lsphp55
PHP Warning:  Unknown: It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected the timezone 'UTC' for now, but please set date.timezone to select your timezone. in Unknown on line 0
session.gc_divisor => 1000 => 1000
session.gc_maxlifetime => 1440 => 1440
session.gc_probability => 1 => 1
session.save_handler => files => files
session.save_path => /var/lib/php/session => /var/lib/php/session

PHP lsphp56
PHP Warning:  Unknown: It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected the timezone 'UTC' for now, but please set date.timezone to select your timezone. in Unknown on line 0
session.gc_divisor => 1000 => 1000
session.gc_maxlifetime => 1440 => 1440
session.gc_probability => 1 => 1
session.save_handler => files => files
session.save_path => /var/lib/php/session => /var/lib/php/session

PHP lsphp70
session.gc_divisor => 1000 => 1000
session.gc_maxlifetime => 1440 => 1440
session.gc_probability => 1 => 1
session.save_handler => files => files
session.save_path => /var/lib/php/session => /var/lib/php/session

PHP lsphp71
session.gc_divisor => 1000 => 1000
session.gc_maxlifetime => 1440 => 1440
session.gc_probability => 1 => 1
session.save_handler => files => files
session.save_path => /var/lib/php/session => /var/lib/php/session

PHP lsphp72
session.gc_divisor => 1000 => 1000
session.gc_maxlifetime => 1440 => 1440
session.gc_probability => 1 => 1
session.save_handler => files => files
session.save_path => /var/lib/php/session => /var/lib/php/session

PHP lsphp73
session.gc_divisor => 1000 => 1000
session.gc_maxlifetime => 1440 => 1440
session.gc_probability => 1 => 1
session.save_handler => files => files
session.save_path => /var/lib/lsphp/session => /var/lib/lsphp/session

PHP lsphp74
session.gc_divisor => 1000 => 1000
session.gc_maxlifetime => 1440 => 1440
session.gc_probability => 1 => 1
session.save_handler => files => files
session.save_path => no value => no value
[root@cybermail:~]#

[root@wcloud:~]# stat /var/lib/lsphp/session/
  File: ‘/var/lib/lsphp/session/’
  Size: 172032          Blocks: 344        IO Block: 4096   directory
Device: fd01h/64769d    Inode: 257949      Links: 2
Access: (1733/drwx-wx-wt)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2020-09-05 03:09:01.874367173 -0400
Modify: 2020-08-26 14:13:35.532083976 -0400
Change: 2020-08-26 14:13:35.532083976 -0400
 Birth: -
[root@wcloud:~]#

What's more fascinating is that when unset and even set in some cases it looks like its using /tmp on both Centos and Ubuntu according to this

[root@cybermail:~]# for version in $(ls /usr/local/lsws|grep lsphp); do echo ""; echo "PHP $version"; /usr/local/lsws/${version}/bin/php -i |grep -Ei 'session.save_handler|session.save_path|session.gc_divisor|session.gc_maxlifetime|session.gc_probability'; done

PHP lsphp53
PHP Warning:  Unknown: It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected 'America/New_York' for 'EDT/-4.0/DST' instead in Unknown on line 0
session.gc_divisor => 1000 => 1000
session.gc_maxlifetime => 1440 => 1440
session.gc_probability => 1 => 1
session.save_handler => files => files
session.save_path => no value => no value

PHP lsphp54
PHP Warning:  Unknown: It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected the timezone 'UTC' for now, but please set date.timezone to select your timezone. in Unknown on line 0
session.gc_divisor => 1000 => 1000
session.gc_maxlifetime => 1440 => 1440
session.gc_probability => 1 => 1
session.save_handler => files => files
session.save_path => /var/lib/php/session => /var/lib/php/session

PHP lsphp55
PHP Warning:  Unknown: It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected the timezone 'UTC' for now, but please set date.timezone to select your timezone. in Unknown on line 0
session.gc_divisor => 1000 => 1000
session.gc_maxlifetime => 1440 => 1440
session.gc_probability => 1 => 1
session.save_handler => files => files
session.save_path => /var/lib/php/session => /var/lib/php/session

PHP lsphp56
PHP Warning:  Unknown: It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected the timezone 'UTC' for now, but please set date.timezone to select your timezone. in Unknown on line 0
session.gc_divisor => 1000 => 1000
session.gc_maxlifetime => 1440 => 1440
session.gc_probability => 1 => 1
session.save_handler => files => files
session.save_path => /var/lib/php/session => /var/lib/php/session

PHP lsphp70
session.gc_divisor => 1000 => 1000
session.gc_maxlifetime => 1440 => 1440
session.gc_probability => 1 => 1
session.save_handler => files => files
session.save_path => /var/lib/php/session => /var/lib/php/session

PHP lsphp71
session.gc_divisor => 1000 => 1000
session.gc_maxlifetime => 1440 => 1440
session.gc_probability => 1 => 1
session.save_handler => files => files
session.save_path => /var/lib/php/session => /var/lib/php/session

PHP lsphp72
session.gc_divisor => 1000 => 1000
session.gc_maxlifetime => 1440 => 1440
session.gc_probability => 1 => 1
session.save_handler => files => files
session.save_path => /var/lib/php/session => /var/lib/php/session

PHP lsphp73
session.gc_divisor => 1000 => 1000
session.gc_maxlifetime => 1440 => 1440
session.gc_probability => 1 => 1
session.save_handler => files => files
session.save_path => /var/lib/lsphp/session => /var/lib/lsphp/session

PHP lsphp74
session.gc_divisor => 1000 => 1000
session.gc_maxlifetime => 1440 => 1440
session.gc_probability => 1 => 1
session.save_handler => files => files
session.save_path => no value => no value
[root@cybermail:~]# for version in $(ls /usr/local/lsws|grep lsphp); do echo ""; echo "PHP $version"; /usr/local/lsws/${version}/bin/php -r 'echo sys_get_temp_dir(), "\n";' ; done

PHP lsphp53
/tmp

PHP lsphp54
/tmp

PHP lsphp55
/tmp

PHP lsphp56
/tmp

PHP lsphp70
/tmp

PHP lsphp71
/tmp

PHP lsphp72
/tmp

PHP lsphp73
/tmp

PHP lsphp74
/tmp
[root@cybermail:~]#

If we went the route where each PHP version had its own subdirectory like other panels we could use something like this

https://stackoverflow.com/questions/654310/cleanup-php-session-files

Propose we set this value to one of the below for all PHP versions in the PHP.ini for clarity /var/lib/lsphp/session/${lsphpver}

Like one of these session.save_path = "/var/lib/lsphp/session/${lsphpver}"

/var/lib/lsphp/session/lsphp53 /var/lib/lsphp/session/lsphp54 /var/lib/lsphp/session/lsphp55 /var/lib/lsphp/session/lsphp56 /var/lib/lsphp/session/lsphp70 /var/lib/lsphp/session/lsphp71 /var/lib/lsphp/session/lsphp72 /var/lib/lsphp/session/lsphp73 /var/lib/lsphp/session/lsphp74

Create the session path directories and chmod it for security to 1733 like the existing one is.

mkdir -p /var/lib/lsphp/session/lsphp{53,54,55,56,70,71,72,73,74}
chmod -R 1733 /var/lib/lsphp/session/lsphp{53,54,55,56,70,71,72,73,74}

For more information on 1733 https://chmodcommand.com/chmod-1733/

Then to fix up the unset or set values we could do the below.

Ubuntu

for phpver in $(ls -1 /usr/local/lsws/ |grep lsphp | sed 's/lsphp//g') ; do echo ""; echo "LSPHP $phpver" ; lsphpver=$(echo $phpver | sed 's/^\(.\{1\}\)/\1./'); sed -i -e "s|^;session.save_path.*|session.save_path = '/var/lib/lsphp/session/lsphp${phpver}'|g" -e "s|^session.save_path.*|session.save_path = '/var/lib/lsphp/session/lsphp${phpver}'|g" /usr/local/lsws/lsphp${phpver}/etc/php/${lsphpver}/litespeed/php.ini ; /usr/local/lsws/lsphp${phpver}/bin/php -i |grep -Ei 'session.save_path' && echo "" ; done; service lsws restart; killall lsphp;

Centos

for version in $(ls /usr/local/lsws|grep lsphp); do echo ""; echo "PHP $version"; sed -i -e "s|^;session.save_path.*|session.save_path = '/var/lib/lsphp/session/${version}'|g" -e "s|^session.save_path.*|session.save_path = '/var/lib/lsphp/session/${version}'|g" /usr/local/lsws/${version}/etc/php.ini; /usr/local/lsws/${version}/bin/php -i |grep -Ei 'date.timezone' && echo "" ; done; service lsws restart; killall lsphp;

Setup a cron to clear stuff older then session.gc_maxlifetime currently set in the php.ini for each version

for version in $(ls /usr/local/lsws|grep lsphp); do echo ""; echo "PHP $version"; session_time=$(/usr/local/lsws/${version}/bin/php -i |grep -Ei 'session.gc_maxlifetime'| grep -Eo "[[:digit:]]+"|sort -u); find -O3 "/var/lib/lsphp/session/${version}" -ignore_readdir_race -depth -mindepth 1 -name 'sess_*' -type f -cmin "${session_time}" -delete; done

Set to a 4 hour default as the 24 min default is kinda low and logs people out too often and as a global default in shared scenario its hard for clients to know how to override this while working in their admin area backends etc. session.gc_maxlifetime = "14400"

This works for both Centos/Ubuntu

grep -Eilr '^memory_limit' --include=\*php.ini /usr/local/lsws/lsphp* | xargs sed -i -e "s/^session.gc_maxlifetime.*/session.gc_maxlifetime = '14400'/g"

This would setup per lsphp version folder in standard path for both OS types and also fix the current and future INI on install for both OS types despite the different path locations for the php.ini

@usmannasir please review this and let me know if you have any questions.

Base php.ini https://github.com/usmannasir/cyberpanel/blob/stable/install/php-configs/php.ini

Resources/References: https://www.php.net/manual/en/function.session-save-path.php https://www.litespeedtech.com/support/forum/threads/change-session-directory.6682/ https://forum.openlitespeed.org/threads/where-on-earth-are-ols-wordpress-vhost-php-session-files-going.3879/

meramsey commented 4 years ago

Wrote a script to do this and setup the cron. https://github.com/usmannasir/cyberpanel/blob/v2.0.3-dev/CPScripts/setup_php_sessions.sh

This should correct the issue and only install the cron if it does not already exist. The other commands are safe to run repeatedly without breaking stuff so i believe it should be like upgrade safe.

@usmannasir @DaWe35 feel free to test it out

wget https://github.com/usmannasir/cyberpanel/raw/v2.0.3-dev/CPScripts/setup_php_sessions.sh && bash setup_php_sessions.sh ;

DaWe35 commented 4 years ago

Hi @whattheserver I just wanted to check your link again and it is now broken

DaWe35 commented 3 years ago

Running this command daily solves the problem for me: find /tmp -name 'sess_*' -mtime +5 -exec rm {} \;

meramsey commented 3 years ago

Hi @whattheserver I just wanted to check your link again and it is now broken

Its in main cyberpanel/ CPScripts/setup_php_sessions.sh

https://github.com/usmannasir/cyberpanel/blob/stable/CPScripts/setup_php_sessions.sh

usmannasir / cyberpanel

503 error, /tmp full with PHP sessions #430