reproducible benchmarks with Docker

stevehu commented 7 years ago

This is migrated from light-example-4j and the original issue is linked below.

We need to find a way to dockerize these benchmark implementations so that everyone can run them on their own hardware.

https://github.com/networknt/light-example-4j/issues/18

h-r-k-matsumoto commented 5 years ago

I am not good at English.I apologize if there is an inappropriate expression.

Is something you want like the one below? https://github.com/networknt/microservices-framework-benchmark/compare/master...h-r-k-matsumoto:fix/master/on-kubernetes

This is using Docker for Mac (using Kubernetes) , it can be reproduced with specific resource limitation. And this communicates with localhost without using kubernetes proxy.

This sample is benchmark reproducible by executing the following.

$ docker build . -t wrk:1.0.0
$ mvn -f light-4j/pom.xml compile jib:dockerBuild
$ kubectl apply -f light-4j/deployments.yaml
$ kubectl exec ${pod-name} -c wrk  --  wrk -t4 -c128 -d30s http://localhost:8080 -s pipeline.lua --latency -- / 16

I think you would not like it, but there are examples of using Kubernetes Service. https://github.com/networknt/microservices-framework-benchmark/compare/master...h-r-k-matsumoto:fix/master/k8s_reproduce_benchmark

If This is fix to the solution, I also make other case Dockerfile, Kubernetes Manifest file.

stevehu commented 5 years ago

@h-r-k-matsumoto Thanks a lot for the effort. My original thought is to dockerize it only without deploying to the Kubernetes cluster it cannot support the load for these high-performance servers. If Kubernetes is used, we need Consul for registry and discovery to keep the millions of requests per second.

I am wondering if it is possible to just use docker without Kubernetes.

Thanks.

h-r-k-matsumoto commented 5 years ago

@stevehu Thank you for your reply. If you cannot use Kubernetes, I think it will be as follows.

$ docker build . -t wrk:1.0.0
$ mvn -f light-4j/pom.xml compile jib:dockerBuild
$ docker network create benchmark
$ docker run --rm --cpus="2.0" -m="400m"  --memory-swap=0 --net=benchmark --name=plaintext -p8080:8080 plaintext:1.0.0
$ docker run --rm --net benchmark wrk:1.0.0   -t4 -c128 -d30s http://plaintext:8080 -s pipeline.lua --latency -- / 16
Running 30s test @ http://plaintext:8080
  4 threads and 128 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     5.01ms    4.19ms  60.15ms   83.84%
    Req/Sec    69.07k    14.56k  120.64k    72.03%
  Latency Distribution
     50%    4.10ms
     75%    6.45ms
     90%    9.60ms
     99%   20.62ms
  8236096 requests in 30.07s, 0.96GB read
Requests/sec: 273859.48
Transfer/sec:     32.65MB
$

If swarm mode is not a performance problem, you can use docker-compose.

Additional notes:2018/11/26 12:19 UTC I'm sorry. This way was discussed in https://github.com/networknt/light-example-4j/issues/18.

networknt / microservices-framework-benchmark

reproducible benchmarks with Docker #28