search

microsoft / reverse-proxy

A toolkit for developing high-performance HTTP reverse proxy applications.
https://microsoft.github.io/reverse-proxy
MIT License
8.61k stars 844 forks source link

Add connection/stream count selections to the benchmarks/dashboard #2563

Open samsp-msft opened 4 months ago

samsp-msft commented 4 months ago

I thinking a bit more about the YARP results compared to NGINX, in chatting with @sebastienros, crank is controlling the incomming rate based on a fixed number of connections, and then streams within those connections. We get different results according to the number of connections that are thrown at us. Based on http/1.1 with a 1024 byte payload we get the following results:

Description application.benchmarks/cpu application.benchmarks/working-set load.http/requests load.http/latency/50 load.http/latency/75 load.http/latency/90 load.http/latency/95 load.http/rps/mean load.http/rps/max
connections_64 82 100 3,262,614 0.244 0.301 0.434 0.618 217,558 267,052
connections_128 85 106 3,863,221 0.364 0.566 0.923 1.146 257,689 374,230
connections_256 90 126 4,282,911 0.649 1.049 1.775 2.15 285,578 433,629
connections_512 88 223 4,563,602 1.263 2.322 3.184 3.754 305,524 417,300
connections_1024 90 1400 4,639,562 2.442 4.106 6.471 9.321 311,574 407,613

You can feed more requests at YARP at once, and get a better mean RPS, at the cost of request latency and working set. We should try some different values for each combination and see what effect that has on the RPS, latency and working set. This should apply to the other proxies as well as they may perform better/worse based on the load.

doddgu commented 3 months ago

Where can I see your test code?

BTW, it's good to see that the focus on high cpu usage issues has increased recently. This shows that there are some high-traffic services that are starting to use yarp. I have reported this issue in #2427 and hope to raise this priority.

MihaZupan commented 3 months ago

The tested YARP code in question is here: https://github.com/microsoft/reverse-proxy/tree/main/testassets/BenchmarkApp With the crank configuration here: https://github.com/aspnet/Benchmarks/blob/main/scenarios/proxy.benchmarks.yml

doddgu commented 3 months ago

The tested YARP code in question is here: https://github.com/microsoft/reverse-proxy/tree/main/testassets/BenchmarkApp With the crank configuration here: https://github.com/aspnet/Benchmarks/blob/main/scenarios/proxy.benchmarks.yml

Thanks!