tsenart / vegeta

HTTP load testing tool and library. It's over 9000!
http://godoc.org/github.com/tsenart/vegeta/lib
MIT License
23.51k stars 1.36k forks source link

Units in JSON report are not consistent with normal report #537

Closed kiran94 closed 3 years ago

kiran94 commented 4 years ago

Version and Runtime

Version: 12.8.3
Commit: d9b795aec8585a0fb435072f68d842d596c332de
Runtime: go1.14 windows/amd64
Date: 2020-03-25T11:03:42Z"

Expected Behaviour

Values in the JSON Report to be be consistent with the normal report

Actual Behaviour

Decimal place is dropped

Steps to Reproduce

  1. Run an Attack to generate some results

  2. Create a normal report: cat output\is_available_all_found_.bin | vegeta report

    Requests      [total, rate, throughput]         1500, 50.04, 49.87
    Duration      [total, attack, wait]             30.081s, 29.978s, 102.998ms
    Latencies     [min, mean, 50, 90, 95, 99, max]  97.963ms, 110.717ms, 102.002ms, 136.545ms, 166.059ms, 208.712ms, 282ms
    Bytes In      [total, mean]                     70500, 47.00
    Bytes Out     [total, mean]                     124500, 83.00
    Success       [ratio]                           100.00%
    Status Codes  [code:count]                      200:1500
    Error Set:
  3. Create a JSON report cat output\is_available_all_found_.bin | vegeta report -type json

{"latencies":{"total":166075974800,"mean":110717316,"50th":102001711,"90th":136545395,"95th":166058847,"99th":208712353,"max":282000200,"min":97962900},"bytes_in":{"total":70500,"mean":47},"bytes_out":{"total":124500,"mean":83},"earliest":"2020-08-29T22:56:04.2008324+01:00","latest":"2020-08-29T22:56:34.1789826+01:00","end":"2020-08-29T22:56:34.2819809+01:00","duration":29978150200,"wait":102998300,"requests":1500,"rate":50.036442875651474,"throughput":49.86511735082189,"success":1,"status_codes":{"200":1500},"errors":[]}

Take mean latency for example, in the normal report it is 110.717ms and in the JSON report it is 110717316 so the decimal place was dropped but I would have expected both reports to be consistent in units. I'm not sure if this is intentional but feel free to close if so!

Additional Context

N/A

surenderssm commented 3 years ago

vegeta report -type=json emits durations always in nanoseconds (machine readable). vegeta report emits durations in human readable units (micro-second, milli-second,second,...)

https://github.com/tsenart/vegeta/blob/master/README.md#report--typejson

kiran94 commented 3 years ago

@surenderssm Happy to close this if you feel that it's the users responsibility to take the machine readable output from when json is set to human readable and it shouldn't be an additional flag.

Thanks

surenderssm commented 3 years ago

Looks like yes to me. @tsenart any other thoughts ?

tsenart commented 3 years ago

I'm not sure if this is intentional but feel free to close if so!

Yep, it's intentional. We don't want to loose precision in the JSON report since it's not for direct human consumption, unlike the text report.