Aerospike client 3 time slower than legacy client

[fbriois]

Hi,

When I run a benchmark, it appears the php-client is more than 3 time slower than the legacy client.

I am using php8.2 with php-client v1.0.2 and on other hand php7.3 with the legacy client https://github.com/aerospike-community/aerospike-client-php.

Clients and server are also in the same server and I use this unmodified configuration for the connection manager: https://github.com/aerospike/php-client/blob/main/aerospike-connection-manager/asld.toml

I use the same configuration between both php, so the main difference is the php version, the aerospike client and the connector (which is written in Rust vs C, I presume).

Here you can find a benchmark result (based from https://github.com/aerospike/php-client/tree/main/benchmark):

$ ./vendor/bin/phpbench run benchmark/ --report=default

PHPBench (1.2.15) running benchmarks... #standwithukraine
with configuration file: /home/ubuntu/php-client/phpbench.json
with PHP version 8.2.18, xdebug ❌, opcache ✔

\Aerospike\AerospikeBenchmark

    benchGetString1.........................I4 - Mo862.710μs (±9.69%)
    benchGetString10........................I4 - Mo752.187μs (±8.31%)
    benchGetString100.......................I4 - Mo723.415μs (±15.13%)
    benchGetString1000......................I4 - Mo572.461μs (±8.73%)
    benchGetString10000.....................I4 - Mo582.190μs (±15.57%)
    benchGetInteger32.......................I4 - Mo611.328μs (±13.72%)
    benchGetInteger64.......................I4 - Mo607.967μs (±16.52%)
    benchPutBins............................I4 - Mo1.559ms (±8.24%)
    benchPutInt64...........................I4 - Mo657.701μs (±11.34%)
    benchPutString1.........................I4 - Mo672.597μs (±12.83%)
    benchPutString10........................I4 - Mo627.837μs (±10.70%)
    benchPutString100.......................I4 - Mo673.821μs (±23.65%)
    benchPutString1000......................I4 - Mo767.200μs (±15.60%)
    benchPutString10000.....................I4 - Mo882.097μs (±14.35%)
    benchPutString100000....................I4 - Mo1.967ms (±7.51%)

Subjects: 15, Assertions: 0, Failures: 0, Errors: 0

+------+--------------------+----------------------+-----+-------+-------------+-------------+--------------+----------------+
| iter | benchmark          | subject              | set | revs  | mem_peak    | time_avg    | comp_z_value | comp_deviation |
+------+--------------------+----------------------+-----+-------+-------------+-------------+--------------+----------------+
| 0    | AerospikeBenchmark | benchGetString1      |     | 10000 | 1,572,256b  | 888.811μs   | -0.28σ       | -2.66%         |
| 1    | AerospikeBenchmark | benchGetString1      |     | 10000 | 1,572,256b  | 981.865μs   | +0.78σ       | +7.53%         |
| 2    | AerospikeBenchmark | benchGetString1      |     | 10000 | 1,572,256b  | 807.754μs   | -1.19σ       | -11.54%        |
| 3    | AerospikeBenchmark | benchGetString1      |     | 10000 | 1,572,256b  | 841.558μs   | -0.81σ       | -7.84%         |
| 4    | AerospikeBenchmark | benchGetString1      |     | 10000 | 1,572,256b  | 1,045.741μs | +1.50σ       | +14.52%        |
| 0    | AerospikeBenchmark | benchGetString10     |     | 10000 | 1,572,816b  | 767.065μs   | -0.25σ       | -2.07%         |
| 1    | AerospikeBenchmark | benchGetString10     |     | 10000 | 1,572,816b  | 803.641μs   | +0.31σ       | +2.60%         |
| 2    | AerospikeBenchmark | benchGetString10     |     | 10000 | 1,572,816b  | 722.532μs   | -0.93σ       | -7.75%         |
| 3    | AerospikeBenchmark | benchGetString10     |     | 10000 | 1,572,816b  | 898.738μs   | +1.77σ       | +14.74%        |
| 4    | AerospikeBenchmark | benchGetString10     |     | 10000 | 1,572,816b  | 724.344μs   | -0.91σ       | -7.52%         |
| 0    | AerospikeBenchmark | benchGetString100    |     | 10000 | 1,572,816b  | 766.442μs   | +0.40σ       | +6.07%         |
| 1    | AerospikeBenchmark | benchGetString100    |     | 10000 | 1,572,816b  | 719.840μs   | -0.03σ       | -0.38%         |
| 2    | AerospikeBenchmark | benchGetString100    |     | 10000 | 1,572,816b  | 683.218μs   | -0.36σ       | -5.45%         |
| 3    | AerospikeBenchmark | benchGetString100    |     | 10000 | 1,572,816b  | 889.547μs   | +1.53σ       | +23.10%        |
| 4    | AerospikeBenchmark | benchGetString100    |     | 10000 | 1,572,816b  | 553.999μs   | -1.54σ       | -23.33%        |
| 0    | AerospikeBenchmark | benchGetString1000   |     | 10000 | 1,572,816b  | 565.938μs   | -0.82σ       | -7.13%         |
| 1    | AerospikeBenchmark | benchGetString1000   |     | 10000 | 1,572,816b  | 556.251μs   | -1.00σ       | -8.72%         |
| 2    | AerospikeBenchmark | benchGetString1000   |     | 10000 | 1,572,816b  | 658.085μs   | +0.92σ       | +7.99%         |
| 3    | AerospikeBenchmark | benchGetString1000   |     | 10000 | 1,572,816b  | 687.733μs   | +1.47σ       | +12.85%        |
| 4    | AerospikeBenchmark | benchGetString1000   |     | 10000 | 1,572,816b  | 578.972μs   | -0.57σ       | -4.99%         |
| 0    | AerospikeBenchmark | benchGetString10000  |     | 10000 | 1,572,816b  | 581.950μs   | -0.49σ       | -7.71%         |
| 1    | AerospikeBenchmark | benchGetString10000  |     | 10000 | 1,572,816b  | 570.012μs   | -0.62σ       | -9.60%         |
| 2    | AerospikeBenchmark | benchGetString10000  |     | 10000 | 1,572,816b  | 614.925μs   | -0.16σ       | -2.48%         |
| 3    | AerospikeBenchmark | benchGetString10000  |     | 10000 | 1,572,816b  | 823.551μs   | +1.97σ       | +30.61%        |
| 4    | AerospikeBenchmark | benchGetString10000  |     | 10000 | 1,572,816b  | 562.247μs   | -0.70σ       | -10.83%        |
| 0    | AerospikeBenchmark | benchGetInteger32    |     | 10000 | 1,572,816b  | 606.482μs   | -0.54σ       | -7.42%         |
| 1    | AerospikeBenchmark | benchGetInteger32    |     | 10000 | 1,572,816b  | 588.429μs   | -0.74σ       | -10.17%        |
| 2    | AerospikeBenchmark | benchGetInteger32    |     | 10000 | 1,572,816b  | 636.399μs   | -0.21σ       | -2.85%         |
| 3    | AerospikeBenchmark | benchGetInteger32    |     | 10000 | 1,572,816b  | 832.208μs   | +1.97σ       | +27.04%        |
| 4    | AerospikeBenchmark | benchGetInteger32    |     | 10000 | 1,572,816b  | 611.879μs   | -0.48σ       | -6.59%         |
| 0    | AerospikeBenchmark | benchGetInteger64    |     | 10000 | 1,572,816b  | 605.118μs   | -0.53σ       | -8.75%         |
| 1    | AerospikeBenchmark | benchGetInteger64    |     | 10000 | 1,572,816b  | 595.550μs   | -0.62σ       | -10.20%        |
| 2    | AerospikeBenchmark | benchGetInteger64    |     | 10000 | 1,572,816b  | 690.887μs   | +0.25σ       | +4.18%         |
| 3    | AerospikeBenchmark | benchGetInteger64    |     | 10000 | 1,572,816b  | 864.656μs   | +1.84σ       | +30.38%        |
| 4    | AerospikeBenchmark | benchGetInteger64    |     | 10000 | 1,572,816b  | 559.615μs   | -0.95σ       | -15.61%        |
| 0    | AerospikeBenchmark | benchPutBins         |     | 10000 | 18,402,376b | 1,385.265μs | -1.47σ       | -12.15%        |
| 1    | AerospikeBenchmark | benchPutBins         |     | 10000 | 18,402,376b | 1,709.142μs | +1.02σ       | +8.39%         |
| 2    | AerospikeBenchmark | benchPutBins         |     | 10000 | 18,402,376b | 1,504.803μs | -0.55σ       | -4.57%         |
| 3    | AerospikeBenchmark | benchPutBins         |     | 10000 | 18,402,376b | 1,733.074μs | +1.20σ       | +9.90%         |
| 4    | AerospikeBenchmark | benchPutBins         |     | 10000 | 18,402,376b | 1,552.211μs | -0.19σ       | -1.57%         |
| 0    | AerospikeBenchmark | benchPutInt64        |     | 10000 | 18,367,256b | 696.816μs   | +0.04σ       | +0.51%         |
| 1    | AerospikeBenchmark | benchPutInt64        |     | 10000 | 18,367,256b | 613.566μs   | -1.01σ       | -11.50%        |
| 2    | AerospikeBenchmark | benchPutInt64        |     | 10000 | 18,367,256b | 841.371μs   | +1.88σ       | +21.36%        |
| 3    | AerospikeBenchmark | benchPutInt64        |     | 10000 | 18,367,256b | 656.512μs   | -0.47σ       | -5.30%         |
| 4    | AerospikeBenchmark | benchPutInt64        |     | 10000 | 18,367,256b | 658.174μs   | -0.45σ       | -5.06%         |
| 0    | AerospikeBenchmark | benchPutString1      |     | 10000 | 18,367,296b | 626.856μs   | -0.98σ       | -12.62%        |
| 1    | AerospikeBenchmark | benchPutString1      |     | 10000 | 18,367,296b | 658.842μs   | -0.64σ       | -8.16%         |
| 2    | AerospikeBenchmark | benchPutString1      |     | 10000 | 18,367,296b | 886.561μs   | +1.84σ       | +23.59%        |
| 3    | AerospikeBenchmark | benchPutString1      |     | 10000 | 18,367,296b | 738.296μs   | +0.23σ       | +2.92%         |
| 4    | AerospikeBenchmark | benchPutString1      |     | 10000 | 18,367,296b | 676.230μs   | -0.45σ       | -5.73%         |
| 0    | AerospikeBenchmark | benchPutString10     |     | 10000 | 18,367,352b | 627.949μs   | -0.70σ       | -7.50%         |
| 1    | AerospikeBenchmark | benchPutString10     |     | 10000 | 18,367,352b | 617.066μs   | -0.85σ       | -9.10%         |
| 2    | AerospikeBenchmark | benchPutString10     |     | 10000 | 18,367,352b | 789.066μs   | +1.52σ       | +16.23%        |
| 3    | AerospikeBenchmark | benchPutString10     |     | 10000 | 18,367,352b | 742.627μs   | +0.88σ       | +9.39%         |
| 4    | AerospikeBenchmark | benchPutString10     |     | 10000 | 18,367,352b | 617.633μs   | -0.84σ       | -9.02%         |
| 0    | AerospikeBenchmark | benchPutString100    |     | 10000 | 18,367,464b | 669.673μs   | -0.53σ       | -12.45%        |
| 1    | AerospikeBenchmark | benchPutString100    |     | 10000 | 18,367,464b | 618.623μs   | -0.81σ       | -19.13%        |
| 2    | AerospikeBenchmark | benchPutString100    |     | 10000 | 18,367,464b | 1,104.944μs | +1.88σ       | +44.45%        |
| 3    | AerospikeBenchmark | benchPutString100    |     | 10000 | 18,367,464b | 795.396μs   | +0.17σ       | +3.98%         |
| 4    | AerospikeBenchmark | benchPutString100    |     | 10000 | 18,367,464b | 636.082μs   | -0.71σ       | -16.85%        |
| 0    | AerospikeBenchmark | benchPutString1000   |     | 10000 | 18,368,584b | 680.349μs   | -0.88σ       | -13.73%        |
| 1    | AerospikeBenchmark | benchPutString1000   |     | 10000 | 18,368,584b | 633.816μs   | -1.26σ       | -19.63%        |
| 2    | AerospikeBenchmark | benchPutString1000   |     | 10000 | 18,368,584b | 979.801μs   | +1.55σ       | +24.24%        |
| 3    | AerospikeBenchmark | benchPutString1000   |     | 10000 | 18,368,584b | 845.254μs   | +0.46σ       | +7.18%         |
| 4    | AerospikeBenchmark | benchPutString1000   |     | 10000 | 18,368,584b | 804.099μs   | +0.13σ       | +1.96%         |
| 0    | AerospikeBenchmark | benchPutString10000  |     | 10000 | 18,379,592b | 923.073μs   | -0.16σ       | -2.31%         |
| 1    | AerospikeBenchmark | benchPutString10000  |     | 10000 | 18,379,592b | 1,206.134μs | +1.93σ       | +27.65%        |
| 2    | AerospikeBenchmark | benchPutString10000  |     | 10000 | 18,379,592b | 821.057μs   | -0.91σ       | -13.11%        |
| 3    | AerospikeBenchmark | benchPutString10000  |     | 10000 | 18,379,592b | 863.174μs   | -0.60σ       | -8.65%         |
| 4    | AerospikeBenchmark | benchPutString10000  |     | 10000 | 18,379,592b | 911.098μs   | -0.25σ       | -3.58%         |
| 0    | AerospikeBenchmark | benchPutString100000 |     | 10000 | 18,469,704b | 1,977.155μs | +0.81σ       | +6.07%         |
| 1    | AerospikeBenchmark | benchPutString100000 |     | 10000 | 18,469,704b | 1,706.905μs | -1.12σ       | -8.43%         |
| 2    | AerospikeBenchmark | benchPutString100000 |     | 10000 | 18,469,704b | 1,997.532μs | +0.95σ       | +7.16%         |
| 3    | AerospikeBenchmark | benchPutString100000 |     | 10000 | 18,469,704b | 1,680.399μs | -1.31σ       | -9.85%         |
| 4    | AerospikeBenchmark | benchPutString100000 |     | 10000 | 18,469,704b | 1,958.178μs | +0.67σ       | +5.05%         |
+------+--------------------+----------------------+-----+-------+-------------+-------------+--------------+----------------+

And here you can find the benchmark result (from https://github.com/aerospike-community/aerospike-client-php/tree/master/examples/performance):

$ php7.3 write.php --host=127.0.0.1 --num-ops=10000 --write-every=10
Connecting to the host ≻ [✓] 
Write 10000 records ≻ [✓] 
10000 sequential writes
Failed writes: 0
Total time: 2.1376039981842s TPS:4678.6027760499

$ php7.3 read.php --host=127.0.0.1 --num-ops=10000 --write-every=10
Connecting to the host ≻ [✓] 
Assuming that write.php was run before to create 10000 records in test.performance
Read 10000 records ≻ [✓] 
10000 sequential reads
Failed reads: 0
Total time: 1.8172008991241s TPS:5503.5191787657

So ~ 182μs to read operation and ~ 214μs to write operation

[khaf]

The new PHP client works in tandem with the connection manager to pool connections for people who heavily use php-fpm in production and other comparable solutions. As a result, it needs to do IPC and that adds to the latency. While the client is slower, it still does its work reliably in under a millisecond. We don't expect that to impede real world applications. Does that affect yours?

[fbriois]

Hi @khaf,

Actually we get a lot of timeout with the older client (too many request in the same time I presume) and we decide to upgrade php-fpm. So we are looking very closely at this client as aerospike is heavily used in production.

I think this client is fast enough to manage all our traffic, but we still need to know if it is more reliable.

Thanks for your answer.