kemalcr / kemal

Fast, Effective, Simple Web Framework
https://kemalcr.com
MIT License
3.6k stars 187 forks source link

Wrong benchmark results? #498

Closed makorne closed 5 years ago

makorne commented 5 years ago

I test "Hello World!" example on i7 16G But can't get even 10% of stated 100238 rps on your site.

weighttp -n 10000 -c 100 -t 8 -k "http://localhost:3000/"  
weighttp 0.4 - a lightweight and simple webserver benchmarking tool

starting benchmark...
spawning thread #1: 13 concurrent requests, 1250 total requests
spawning thread #2: 13 concurrent requests, 1250 total requests
spawning thread #3: 13 concurrent requests, 1250 total requests
spawning thread #4: 13 concurrent requests, 1250 total requests
spawning thread #5: 12 concurrent requests, 1250 total requests
spawning thread #6: 12 concurrent requests, 1250 total requests
spawning thread #7: 12 concurrent requests, 1250 total requests
spawning thread #8: 12 concurrent requests, 1250 total requests
progress:  10% done
progress:  20% done
progress:  30% done
progress:  40% done
progress:  50% done
progress:  60% done
progress:  70% done
progress:  80% done
progress:  90% done
progress: 100% done

finished in 1 sec, 61 millisec and 594 microsec, 9419 req/s, 1113 kbyte/s
crystal -v
Crystal 0.26.1 [391785249] (2018-08-27)

LLVM: 4.0.0
Default target: x86_64-unknown-linux-gnu
Linux d 4.9.0-8-amd64 #1 SMP Debian 4.9.110-3+deb9u6 (2018-10-08) x86_64 GNU/Linux

hello.cr:

require "kemal"
module Hello
  VERSION = "0.1.0"
  Kemal.config.env = "production"

  get "/" do
    "Hello World!"
  end

  Kemal.run

end
straight-shoota commented 5 years ago

Did you compile with --release?

makorne commented 5 years ago

Now: crystal run src/hello.cr --release

weighttp -n 10000 -c 100 -t 8 -k "http://localhost:3000/"  
weighttp 0.4 - a lightweight and simple webserver benchmarking tool

starting benchmark...
spawning thread #1: 13 concurrent requests, 1250 total requests
spawning thread #2: 13 concurrent requests, 1250 total requests
spawning thread #3: 13 concurrent requests, 1250 total requests
spawning thread #4: 13 concurrent requests, 1250 total requests
spawning thread #5: 12 concurrent requests, 1250 total requests
progress:  10% done
progress:  20% done
spawning thread #6: 12 concurrent requests, 1250 total requests
spawning thread #7: 12 concurrent requests, 1250 total requests
spawning thread #8: 12 concurrent requests, 1250 total requests
progress:  30% done
progress:  40% done
progress:  50% done
progress:  60% done
progress:  70% done
progress:  80% done
progress:  90% done
progress: 100% done

finished in 0 sec, 779 millisec and 752 microsec, 12824 req/s, 1515 kbyte/s
s0kil commented 5 years ago

@makorne You can get more performance by disabling Kemal logging. Ex:

require "kemal"

logging false

# Set root. If not specified the default content_type is 'text'
get "/" do
  "Hello Kemal!"
end

Kemal.run

Bench:

$ weighttp -n 10000 -c 100 -t 8 -k "http://localhost:3000/"
weighttp 0.4 - a lightweight and simple webserver benchmarking tool

starting benchmark...
spawning thread #1: 13 concurrent requests, 1250 total requests
spawning thread #2: 13 concurrent requests, 1250 total requests
spawning thread #3: 13 concurrent requests, 1250 total requests
spawning thread #4: 13 concurrent requests, 1250 total requests
spawning thread #5: 12 concurrent requests, 1250 total requests
spawning thread #6: 12 concurrent requests, 1250 total requests
spawning thread #7: 12 concurrent requests, 1250 total requests
spawning thread #8: 12 concurrent requests, 1250 total requests
progress:  10% done
progress:  20% done
progress:  30% done
progress:  40% done
progress:  50% done
progress:  60% done
progress:  70% done
progress:  80% done
progress:  90% done
progress: 100% done

finished in 0 sec, 198 millisec and 597 microsec, 50353 req/s, 5949 kbyte/s
requests: 10000 total, 10000 started, 10000 done, 10000 succeeded, 0 failed, 0 errored
status codes: 10000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 1210000 bytes total, 1090000 bytes http, 120000 bytes data

With Logging:

$ weighttp -n 10000 -c 100 -t 8 -k "http://localhost:3000/"
weighttp 0.4 - a lightweight and simple webserver benchmarking tool

starting benchmark...
spawning thread #1: 13 concurrent requests, 1250 total requests
spawning thread #2: 13 concurrent requests, 1250 total requests
spawning thread #3: 13 concurrent requests, 1250 total requests
spawning thread #4: 13 concurrent requests, 1250 total requests
spawning thread #5: 12 concurrent requests, 1250 total requests
spawning thread #6: 12 concurrent requests, 1250 total requests
spawning thread #7: 12 concurrent requests, 1250 total requests
spawning thread #8: 12 concurrent requests, 1250 total requests
progress:  10% done
progress:  20% done
progress:  30% done
progress:  40% done
progress:  50% done
progress:  60% done
progress:  70% done
progress:  80% done
progress:  90% done
progress: 100% done

finished in 0 sec, 317 millisec and 972 microsec, 31449 req/s, 3716 kbyte/s
requests: 10000 total, 10000 started, 10000 done, 10000 succeeded, 0 failed, 0 errored
status codes: 10000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 1210000 bytes total, 1090000 bytes http, 120000 bytes data
sdogruyol commented 5 years ago

Haven't used 'weighttpbefore. All the Kemal HTTP benchmarks were done usingwrk`.

As others stated previously, you can use --release and disable logging to get more performance