Inaccurate Laravel performance due to stack mistakes

[rk]

Hi,

I use Laravel every day, and saw this on Hacker News. So when I saw that the average response time for Laravel was 378.5ms, I wanted to chip in a few comments:

• You use nginx as a referse-forward proxy to php serve. php serve has known performance issues when used for anything outside of development. I don't even use it for that, because of its wildly random performance characteristics.
• You did not use nginx + PHP-FPM to serve the application. This is a typical server setup, because PHP-FPM provides huge performance boosts over CGI.
• You did not disable the xdebug module, which is known to degrade performance 200-500% in performance environments. I have had pages take many seconds, only for them to take 100ms when I turned xdebug off. I think it's because of retaining stack frames, etc.
• You also did not boost opcache.max_accelerated_files up to 40000 or more. Laravel uses a large number of files for the framework, so by keeping the default 10k files limit it won't cache everything with each request. See this.

Note: Ofcourse there are a lot of variables that influence application performance. A lot depends on a web server that you put in front of your application, so numbers above may vary in different configurations. The test was performed with webservers that are typical for the stack just to have a picture of how performance of those frameworks compare to each other.

Which makes the above highlighted section absolutely not true for Laravel. Now, to be clear: how you served the application would have negatively skewed any PHP framework's results. I know it was simple to do, but you went against best practices and even default OS-level installs for PHP hosting. Even a basic Apache + PHP set up will average 50-200ms for Laravel with minor optimizations.

Could you please take another look and try to fix how you're bootstrapping the environment?

[alchaplinsky]

Hey @rk, first of all, thanks for your detailed description! I'd definitely want to do more accurate measurements on this one, so I appreciate your contribution. 2 notes, on how laravel was set up before measurement:

1. Dockerfile here is a bit misleading as it contains CMD php artisan serve, but in fact, I ran Nginx pointing it directly to laravel's public folder (if that's not a common setup I'll gladly update it). I noticed that php serve is slow as hell and it can't be real performance. I haven't managed to set up Nginx with PHP in a docker container, yet. I will still do that and will update the results.

2. Regarding different tweaks and improvements that can be done, to improve the performance of the app, I believe this is true for other frameworks as well. Each o them has some techniques for speeding up, but the goal was to test frameworks as they come out of the box (just by running them in the production environment). That's why I haven't considered changing opcache.max_accelerated_files.

I'd definitely want to eliminate development environment specifics and debug tools. But I'm not sure if xdebug module is on by default for PHP 8? I haven't found a configuration that turns it on or off in the code, except for this ENV variable in docker-compose.yml.

Would you be so kind to give some advice on that?

[rk]

I don't use Docker, so I don't have any Docker specific advice to offer. Sorry. I don't know if this is helpful, but TechEmpower's benchmarks do use Docker and customize the php.ini settings: https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/PHP/laravel/laravel.dockerfile

Using Nginix pointed to the public directory should still be correct usage. I don't know if Nginx uses PHP-FPM by default, but I would guess it does.

Regarding different tweaks and improvements that can be done, to improve the performance of the app, I believe this is true for other frameworks as well. Each o them has some techniques for speeding up, but the goal was to test frameworks as they come out of the box (just by running them in the production environment). That's why I haven't considered changing opcache.max_accelerated_files.

As far as this goes, I would argue that for fair testing of any modern PHP app that uses composer, the following should be done for fair testing:

• Disable xdebug.
• Set opcache.max_accelerated_files to 40000.

This is because with Composer, PHP apps can face the same issue that Node + NPM can -- except worse. Where Node loads all those files, caches the compiled byte code and JITs it once -- PHP has to load the code for every request. This is why PHP-FPM and OPCache are necessary. They allow skipping the reload/recompile part of the process, putting the language closer to the 30ms performance mark.

I think xdebug is supported by PHP 8 now, but I don't think it's enabled by default. But that might depend on your distro's packaging.