php / php-src

The PHP Interpreter
https://www.php.net
Other
38.12k stars 7.74k forks source link

set_timeout() fails to terminate execution when IO Latency is high #11971

Open mikhainin opened 1 year ago

mikhainin commented 1 year ago

Description

I'm not sure if this is a bug or not and I haven't found the proper channel to ask (shall I raise a discussion in Internals?).

We faced a problem in our production cluster. Our setup is following:

SAPI: fpm max_execution_time: 28 hard_timeout: 2

When Opcache is reloading, we're having excessive IO pressure, which is expected - PHP starts reading everything from disk, etc... However, scripts started executing for 5+ minutes - that wasn't expected.

Further investigation showed that zend_set_timeout_ex uses setitimer(ITIMER_PROF) by default, which means the time the process spends in IOWait doesn't seem to count.

So, basically, even though our gateway cannot wait for such long - it expects that application will finish earlier, PHP still performs the request. One more backside of such behaviour - the restart of Opcache takes longer - it waits until all the workers finish execution (or become killed by opcache when opcache.force_restart_timeout is reached).

This problem doesn't seem to appear when we use ITIMER_REAL. I can provide how we tested this.

I can see a couple of solutions here:

KapitanOczywisty commented 1 year ago

There is RFC to add wall time limit https://wiki.php.net/rfc/max_execution_wall_time

mikhainin commented 1 year ago

Oh, that's interesting @KapitanOczywisty! As far as I can see it didn't go forward. Any reason for that? Would it be possible to re-open that activity?

KapitanOczywisty commented 1 year ago

I wasn't following progress of that, but you can read about that: #6504 https://externals.io/message/112492