programatik29 / rust-web-benchmarks

Benchmarking web frameworks written in rust with rewrk tool.
MIT License
248 stars 33 forks source link

Failed to test on Mac systems #2

Closed ecafkoob closed 3 years ago

ecafkoob commented 3 years ago

osinfo:

ProductName: macOS ProductVersion: 11.5.2 BuildVersion: 20G95

kern.maxfilesperproc: 49152 kern.maxfiles: 98304

result:

➜ echo "Results for hyper:" && rewrk -t 12 -c 300 -d 10s -h http://localhost:3000/

Results for hyper: Beginning round 1... Benchmarking 300 connections @ http://localhost:3000/ for 10 second(s) failed to run benchmark round due to error: error combining results: connection closed

use wrk it's ok:

➜ wrk -t12 -c400 -d30s --latency http://localhost:3000/ wrk http://localhost:3000/ Running 30s test @ http://localhost:3000/ 12 threads and 400 connections Thread Stats Avg Stdev Max +/- Stdev Latency 808.44us 247.22us 3.95ms 72.97% Req/Sec 23.51k 9.67k 141.20k 79.11% Latency Distribution 50% 819.00us 75% 0.96ms 90% 1.10ms 99% 1.40ms 7020424 requests in 30.10s, 589.18MB read Socket errors: connect 157, read 89, write 0, timeout 0 Requests/sec: 233235.83 Transfer/sec: 19.57MB

programatik29 commented 3 years ago

This is a rewrk problem. The problem is when an error occurs on one of the connections, program will return an error. So it isn't platform specific. I will open an issue there.

programatik29 commented 3 years ago

Can you try again with latest rewrk?

ecafkoob commented 3 years ago

when i use rewrk -t 12 -c 500 -d 10s -h http://localhost:3000/ hyper crashed with Error: Os { code: 24, kind: Other, message: "Too many open files" }. rewrk -t 12 -c 400 -d 10s -h http://localhost:3000/ is ok. wrk works well with 12 threads and 500 connections.

programatik29 commented 3 years ago

Might be related to this which should be fixed with this PR.

ecafkoob commented 3 years ago
image image

I want to know why there is such a big difference between URL and ip