codesenberg / bombardier

Fast cross-platform HTTP benchmarking tool written in Go
MIT License
5.91k stars 313 forks source link

Add latencies for 2xx responses #73

Open chr1st1ank opened 3 years ago

chr1st1ank commented 3 years ago

In cases where a service returns a lot of error codes it is impossible to know the response times of the successful operations. Now an additional histogram value collection for 2xx responses is added.

Before:

Statistics        Avg      Stdev        Max
  Reqs/sec      2195.55     302.63    2600.64
  Latency        3.67ms    30.53ms      1.02s
  Latency Distribution
     50%     2.90ms
     75%     3.04ms
     90%     3.28ms
     95%     3.63ms
     99%     5.45ms
  HTTP codes:
    1xx - 0, 2xx - 11, 3xx - 0, 4xx - 0, 5xx - 11009
    others - 0
  Throughput:   513.10KB/s

After:

Statistics        Avg      Stdev        Max
  Reqs/sec      1947.65     558.64    2566.71
  Latency        4.13ms    32.36ms      1.01s
  Latency Distribution (Total)
     50%     2.91ms
     75%     3.18ms
     90%     4.08ms
     95%     4.96ms
     99%    10.69ms
  Latency Distribution (2xx)
     50%      1.00s
     75%      1.00s
     90%      1.01s
     95%      1.01s
     99%      1.01s
  HTTP codes:
    1xx - 0, 2xx - 11, 3xx - 0, 4xx - 0, 5xx - 9785
    others - 0
  Throughput:   456.62KB/s

In case of no 2xx the result is omitted:

Statistics        Avg      Stdev        Max
  Reqs/sec      2052.87     469.71    2646.66
  Latency        3.89ms     1.43ms    29.66ms
  Latency Distribution (Total)
     50%     3.69ms
     75%     4.16ms
     90%     5.34ms
     95%     6.36ms
     99%     9.31ms
  HTTP codes:
    1xx - 0, 2xx - 0, 3xx - 0, 4xx - 10271, 5xx - 0
    others - 0
  Throughput:   659.54KB/s

The json output now looks like this:

{
  "spec": {
    "numberOfConnections": 4,
    "testType": "timed",
    "testDurationSeconds": 2,
    "method": "GET",
    "url": "http://localhost:8088/calculate-limited?argument=abc",
    "body": "",
    "stream": false,
    "timeoutSeconds": 1,
    "client": "fasthttp"
  },
  "result": {
    "bytesRead": 596028,
    "bytesWritten": 320988,
    "timeTakenSeconds": 3.512297515,
    "req1xx": 0,
    "req2xx": 3,
    "req3xx": 0,
    "req4xx": 0,
    "req5xx": 3482,
    "others": 0,
    "latency": {
      "mean": 2868.1977044476325,
      "stddev": 52273.760421009916,
      "max": 2505392,
      "percentiles": {
        "50": 1049,
        "75": 1169,
        "90": 1582,
        "95": 1907,
        "99": 2979
      },
      "percentiles2xx": {
        "50": 997482,
        "75": 997482,
        "90": 2505392,
        "95": 2505392,
        "99": 2505392
      }
    },
    "rps": {
      "mean": 1720.3570087831786,
      "stddev": 453.862925891273,
      "max": 2305.700383057032,
      "percentiles": {
        "50": 1773.912226,
        "75": 2167.367572,
        "90": 2259.393315,
        "95": 2276.839843,
        "99": 2289.22986
      }
    }
  }
}

If there are no 2xx responses the new json attribute is empty: "percentiles2xx": {}

chr1st1ank commented 3 years ago

Hey @codesenberg, any chance to get this PR into master? Or anything I can do to make it easier for you to review? I tried to keep the changes minimal, but as the architecture was I had to touch a few files, unfortunately. A more general solution (e.g. a tabular output format with results on all status code ranges) I didn't try because this would need some more serious refactoring first.

chr1st1ank commented 2 years ago

As a preliminary solution I used my forked master branch to publish this feature: https://github.com/chr1st1ank/bombardier/releases/tag/v1.3.5

No offense meant. I just needed windows + linux builds of it on multiple machines. I'd be very happy to merge it into the main repository and get rid of my fork.