search

php / php-src

The PHP Interpreter
https://www.php.net
Other
37.93k stars 7.72k forks source link

Opcache with file_cache and JIT enabled crashes an application #15497

Closed voodooism closed 2 weeks ago

voodooism commented 3 weeks ago

Description

On our production server running a Symfony application with PHP-FPM and Nginx, I've noticed an occasional strange behavior: after an HTTP request, we sometimes receive an empty response with a 200 HTTP status code. In the PHP-FPM logs, I see the following message:

Error: Shared memory lock not obtained

After conducting some tests, I realized that this error occurs only when the file_cache and JIT options are enabled. Additionally, I noticed that the max_accelerated_files option also plays a role in this behavior. Our application is quite large, so this setting is configured to 20,000 on our production server. However, if I turn off the file_cache, everything works fine, even with the max_accelerated_files set to its minimum value (200 instead of 20000). The same happens with JIT turned off - everything works well, with JIT disabled.

To reproduce this behavior locally, I created a simple project: I downloaded the same version of Symfony that we have in production (6.4), built the application using PHP and Nginx images from Docker Hub, and decreased the max_accelerated_files to its minimum value (200, according to the documentation).

Steps to reproduce: First, let's make sure that everything works fine:

git clone git@github.com:voodooism/opcache-testing.git
cd ./opcache-testing
composer install
docker-compose up -d --build

Expected response:

curl http://localhost:8080/health
{"status":"ok"}

Logs from php-fpm container:

[19-Aug-2024 12:49:25] NOTICE: [pool www] 'user' directive is ignored when FPM is not running as root
[19-Aug-2024 12:49:25] NOTICE: [pool www] 'user' directive is ignored when FPM is not running as root
[19-Aug-2024 12:49:25] NOTICE: [pool www] 'group' directive is ignored when FPM is not running as root
[19-Aug-2024 12:49:25] NOTICE: [pool www] 'group' directive is ignored when FPM is not running as root
[19-Aug-2024 12:49:25] NOTICE: fpm is running, pid 1
[19-Aug-2024 12:49:25] NOTICE: ready to handle connections
172.18.0.4 -  19/Aug/2024:12:49:28 +0000 "GET /index.php" 200

Then, let's change the max_accelerated_files

echo "opcache.max_accelerated_files = 200" >> ./docker/php/conf.d/opcache.ini
docker-compose up -d --build

Bad behavior (no response body at all):

curl http://localhost:8080/health

Logs from php-fpm container:

[19-Aug-2024 12:50:38] NOTICE: [pool www] 'user' directive is ignored when FPM is not running as root
[19-Aug-2024 12:50:38] NOTICE: [pool www] 'user' directive is ignored when FPM is not running as root
[19-Aug-2024 12:50:38] NOTICE: [pool www] 'group' directive is ignored when FPM is not running as root
[19-Aug-2024 12:50:38] NOTICE: [pool www] 'group' directive is ignored when FPM is not running as root
[19-Aug-2024 12:50:38] NOTICE: fpm is running, pid 1
[19-Aug-2024 12:50:38] NOTICE: ready to handle connections
Mon Aug 19 12:50:40 2024 (7): Error Shared memory lock not obtained
172.18.0.4 -  19/Aug/2024:12:50:40 +0000 "GET /index.php" 200

Then let's override the existent file_cache option and turn it off:

echo "opcache.file_cache = null" >> ./docker/php/conf.d/opcache.ini

Everything works as expected then:

curl http://localhost:8080/health
{"status":"ok"}

Logs:

[19-Aug-2024 13:45:50] NOTICE: [pool www] 'user' directive is ignored when FPM is not running as root
[19-Aug-2024 13:45:50] NOTICE: [pool www] 'user' directive is ignored when FPM is not running as root
[19-Aug-2024 13:45:50] NOTICE: [pool www] 'group' directive is ignored when FPM is not running as root
[19-Aug-2024 13:45:50] NOTICE: [pool www] 'group' directive is ignored when FPM is not running as root
[19-Aug-2024 13:45:50] NOTICE: fpm is running, pid 1
[19-Aug-2024 13:45:50] NOTICE: ready to handle connections
172.18.0.4 -  19/Aug/2024:13:46:26 +0000 "GET /index.php" 200
More information for context PHP version ``` PHP 8.2.22 (cli) (built: Aug 13 2024 02:05:25) (NTS) Copyright (c) The PHP Group Zend Engine v4.2.22, Copyright (c) Zend Technologies with Zend OPcache v8.2.22, Copyright (c), by Zend Technologies ``` php -m ``` [PHP Modules] Core ctype curl date dom fileinfo filter hash iconv json libxml mbstring mysqlnd openssl pcre PDO pdo_pgsql pdo_sqlite Phar posix random readline Reflection session SimpleXML sodium SPL sqlite3 standard tokenizer xml xmlreader xmlwriter Zend OPcache zlib [Zend Modules] Zend OPcache ``` php -i|grep opcache. ``` /usr/local/etc/php/conf.d/docker-php-ext-opcache.ini, /usr/local/etc/php/conf.d/opcache.ini opcache.blacklist_filename => no value => no value opcache.consistency_checks => 0 => 0 opcache.dups_fix => Off => Off opcache.enable => On => On opcache.enable_cli => Off => Off opcache.enable_file_override => Off => Off opcache.error_log => no value => no value opcache.file_cache => /var/www/html/var => /var/www/html/var opcache.file_cache_consistency_checks => On => On opcache.file_cache_only => Off => Off opcache.file_update_protection => 2 => 2 opcache.force_restart_timeout => 180 => 180 opcache.huge_code_pages => Off => Off opcache.interned_strings_buffer => 16 => 16 opcache.jit => tracing => tracing opcache.jit_bisect_limit => 0 => 0 opcache.jit_blacklist_root_trace => 16 => 16 opcache.jit_blacklist_side_trace => 8 => 8 opcache.jit_buffer_size => 128M => 128M opcache.jit_debug => 0 => 0 opcache.jit_hot_func => 127 => 127 opcache.jit_hot_loop => 64 => 64 opcache.jit_hot_return => 8 => 8 opcache.jit_hot_side_exit => 8 => 8 opcache.jit_max_exit_counters => 8192 => 8192 opcache.jit_max_loop_unrolls => 8 => 8 opcache.jit_max_polymorphic_calls => 2 => 2 opcache.jit_max_recursive_calls => 2 => 2 opcache.jit_max_recursive_returns => 2 => 2 opcache.jit_max_root_traces => 1024 => 1024 opcache.jit_max_side_traces => 128 => 128 opcache.jit_prof_threshold => 0.005 => 0.005 opcache.lockfile_path => /var/www/html/var => /var/www/html/var opcache.log_verbosity_level => 1 => 1 opcache.max_accelerated_files => 200 => 200 opcache.max_file_size => 0 => 0 opcache.max_wasted_percentage => 5 => 5 opcache.memory_consumption => 2048 => 2048 opcache.opt_debug_level => 0 => 0 opcache.optimization_level => 0x7FFEBFFF => 0x7FFEBFFF opcache.preferred_memory_model => no value => no value opcache.preload => no value => no value opcache.preload_user => no value => no value opcache.protect_memory => Off => Off opcache.record_warnings => Off => Off opcache.restrict_api => no value => no value opcache.revalidate_freq => 2 => 2 opcache.revalidate_path => Off => Off opcache.save_comments => On => On opcache.use_cwd => On => On opcache.validate_permission => Off => Off opcache.validate_root => Off => Off opcache.validate_timestamps => Off => Off ```

PHP Version

PHP 8.2.22

Operating System

Ubuntu 22.04.4

iluuu1994 commented 3 weeks ago

/cc @dstogov

dstogov commented 2 weeks ago

This error may be caught only in PHP-8.2. PHP-8.3 debug build should lead to assertion. PHP-8.3 release build misses the error condition because commit 10d43c40dd406178af6ff0dfa14194283e123778 has removed this error message and replaced it with ZEND_ASSERT(), but ZEND_ASSERT() is ignored in release builds. This means PHP-8.3 might do something completely wrong and should be fixed.

dstogov commented 2 weeks ago

The bug should be fixed now. I was able to reproduce it without your application, by setting opcache.max_accelerated_files=200 and opcache.file_cache and running Symfony demo app.

$ php-cgi /.../symfony_demo/public/index.php

Please check if the the fix works for you.

It seems we have some different problem with PCRE cache...