[performance] octane high latency underload (compare to laravel-swoole)

[kocoten1992]

• Octane Version: 0.3
• Laravel Version: 8.37.0
• PHP Version: PHP 8.0.3 (cli)
• Server: Swoole
• Database Driver & Version: not relevant

Description:

Previously, I'm using laravel-swoole, but decide to stick with octane since it will be first class citizen. Conducting a small experiment show that, under heavy load octane response time is significantly higher than laravel-swoole (2ms vs 50ms).

Steps To Reproduce:

cd /tmp/
composer create-project laravel/laravel example-app
cd example-app
composer require laravel/octane
php artisan octane:install # choose swoole
composer require swooletw/laravel-swoole

# disable ALL middleware in app/Http/Kernel.php
# add this route to api.php
Route::get('perf', function () {
    return response()->json();
});

# start octane and laravel-swoole
php artisan octane:start --quiet
php artisan swoole:http start

wrk -c4 -t4 -d5 http://127.0.0.1:8000/api/perf # octane
wrk -c4 -t4 -d5 http://127.0.0.1:1215/api/perf # laravel-swoole

Result

Running 5s test @ http://127.0.0.1:8000/api/perf
  4 threads and 4 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    67.29ms  104.33ms 383.10ms   81.59%
    Req/Sec   427.55    351.40     1.02k    59.13%
  6044 requests in 5.01s, 1.23MB read
Requests/sec:   1206.61
Transfer/sec:    252.16KB

Running 5s test @ http://127.0.0.1:1215/api/perf
  4 threads and 4 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.19ms    2.43ms  42.82ms   96.46%
    Req/Sec     1.21k   161.56     2.06k    72.14%
  24208 requests in 5.10s, 4.50MB read
Requests/sec:   4747.04
Transfer/sec:      0.88MB

The result show that underload octane latency go much higher than it rival (that would translate into lower Requests/sec).

[themsaid]

Octane does a few things differently to eliminate memory leaks and proper Laravel container clearing between requests. This is not the case with the other swoole libraries we saw. The difference in handling clearing between requests could lead to a small overhead.

Also Octane disables coroutines, which is enabled in the other libraries. It's mainly disabled because major changes are needed in the Laravel container as well as most packages to make it work correctly.

[kocoten1992]

@themsaid but even php-fpm latency is not that high underload, the result below is from php-fpm

Running 5s test @ http://127.0.0.1:80/api/perf
  4 threads and 4 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    10.56ms    1.86ms  34.30ms   91.65%
    Req/Sec    95.08      8.22   111.00     87.00%
  1896 requests in 5.01s, 401.71KB read
Requests/sec:    378.74
Transfer/sec:     80.25KB

if latency is so high, that would defeat the point of octane :(

[themsaid]

Are you testing with a fresh laravel app?

[kocoten1992]

@themsaid Yes, I was testing on fresh laravel app, the detail is on first post, I even try on DigitalOcean, the result is the same.

[taylorotwell]

I'm having trouble recreating this on a fresh Laravel application 🤔

[barryvdh]

50ms does seems slow. I haven't compared them but when I run the same test I also get something like 7ms latency, and 3-4 ms times in the console, without any optimisations/caching besides disabling the middleware.

[taylorotwell]

We tracked this down to the APP_ENV not being production. When it is not production, Swoole is going to write NOTICE and WARNING entires to the log file on every request because we set log_level to 1. Make sure you set your application to production. Make sure you are not getting output in your storage/logs/swoole_http.log - if you are, you are going to get high latency. Make sure your config is not cached after making this change.