mnapoli / bref-bootstrap-benchmarks

Benchmark of possible solutions for Bref and the new AWS Lambda runtime API
https://github.com/mnapoli/bref/issues/100
22 stars 3 forks source link

Coldstart overhead #10

Open barryvdh opened 5 years ago

barryvdh commented 5 years ago

We're measuring performance, which is interesting obviously, but perhaps we should also look at the (additional) bootstrap time.

Eg. the php webserver has a sleep of 1 sec, which can be significant if it's run not so often. I'm not sure what realistic numbers are in the percentage of cold starts, but should we take them into account?

We could benchmark them using a fresh deploy with a higher concurrency, right?

So something like ab -c 100 -n 100 https://.. would spawn 100 fresh workers (after a new deploy), right?

staabm commented 5 years ago

IIRC the "old symfony webserver" had some kind of detection when the php webserver started successfully. sometimes 1 sec is not enough, sometimes its a lot faster.

barryvdh commented 5 years ago

Ahh yes, so like this: https://github.com/symfony/web-server-bundle/blob/9f071b61b9b554f675d45e3382e3a07581bea0e8/WebServer.php#L121-L141

Something like:

if (false !== $fp = @fsockopen('127.0.0.1', 8000, $errno, $errstr, 1)) {
    fclose($fp);
    return true;
}

return false;
barryvdh commented 5 years ago

So the init for the built-in server and the fcgi server seems to be something like 300ms - 400ms.

staabm commented 5 years ago

Cool, thx for checking

bubba-h57 commented 5 years ago

If we use the file socket instead, isn't it available as soon as the socket is?

barryvdh commented 5 years ago

I do that for the cgi one, bit this is the builtin server

mnapoli commented 5 years ago

:+1: agreed this could be the next step.

staabm commented 5 years ago

Maybe it would be even worth to profile php’s internal webserver with the goal of improving startup time

barryvdh commented 5 years ago

If you look at stackery's issues, perhaps compatibility with existing code is more important; https://github.com/stackery/php-lambda-layer/issues It seems there are still some issues with the built-in server sometimes. (But that's probably due to their configuration)

mnapoli commented 5 years ago

Agreed, ideally we would have a default mode for Bref that is fast but proven to be reliable, then we can have one or several experimental modes that provide performance improvements. With time we can then see if an experimental mode is actually stable.