search

lenkan / keria-bench

1 stars 0 forks source link

Is docker the best tool to use for benchmarking? #2

Open pfeairheller opened 11 months ago

pfeairheller commented 11 months ago

I'm getting really good results up to -c 500 when running this on "bare metal" MacOS on an M2 and I'm wondering if docker and/or Linux is getting in the way somehow. Sam sent me a TON of information about configuring TCP buffers on Linux to make benchmarks like these actually perform but I haven't had time to dig through it yet.

Here is a sample run.

`(keripy) orion:(feat-direct-exchange)~/git_root/keripy $ ab -r -c 500 -n 2000 http://localhost:3902/spec.yaml This is ApacheBench, Version 2.3 <$Revision: 1903618 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient) Completed 200 requests Completed 400 requests Completed 600 requests Completed 800 requests Completed 1000 requests Completed 1200 requests Completed 1400 requests Completed 1600 requests Completed 1800 requests Completed 2000 requests Finished 2000 requests

Server Software: Ioflo Server Hostname: localhost Server Port: 3902

Document Path: /spec.yaml Document Length: 0 bytes

Concurrency Level: 500 Time taken for tests: 0.120 seconds Complete requests: 2000 Failed requests: 2001 (Connect: 2, Receive: 1999, Length: 0, Exceptions: 0) Total transferred: 24342 bytes HTML transferred: 24204 bytes Requests per second: 16707.88 [#/sec] (mean) Time per request: 29.926 [ms] (mean) Time per request: 0.060 [ms] (mean, across all concurrent requests) Transfer rate: 198.59 [Kbytes/sec] received

Connection Times (ms) min mean[+/-sd] median max Connect: 0 6 5.0 5 18 Processing: 0 4 2.8 3 59 Waiting: 0 0 1.0 0 46 Total: 2 10 7.2 8 59

Percentage of the requests served within a certain time (ms) 50% 8 66% 10 75% 11 80% 21 90% 23 95% 24 98% 25 99% 25 100% 59 (longest request) `

I am able to make ab give up with a "More than 10 errors" message when I run this with -c 10000. I'm looking into this as well.

pfeairheller commented 11 months ago

Looking more at this I already see that ab is tracking read errors differently from socket errors and all of these are failing on read errors thus allowed to complete.