Aggregate results from test runs to CSV

[hannahhoward]

What

At the end of a load tests, we want to aggregate overall results from various test runs into a CSV, where each row contains:

parameters

- Protocol (HTTP, Bitswap)
- Scenario (example for HTTP Full piece / small piece / range)
- Concurrency
- Latency (leave blank for now)
- Bandwidth (leave blank for now)

• results *

  • TTFB (regular, raw, delta between regular + raw) (mean, median, 90%, etc)
  • Bandwidth (regular, raw, delta between regular + raw) (mean, median, 90%, etc)
  • Success rate
  • Missed WindowPost (leave blank for now)
  • CPU (blank unless implemented after #6 )
  • Memory (blank unless implemented after #6 )

  Suggested implementation

  At the end of loadtest.sh, run a program that:

  • reads in the various saved json files
  • digs out the relevant values
  • saves them to a CSV

  Use any programming language you like, though probably recommend node.

  Alternate implementation idea

  dig the various values out directly in handleSummary() the only issue is that return values for handleSummary overwrite previous values you might try reading in the current CSV into a SharedArray during the init context, and then appending only the last row in handle summary before rewriting.

Acceptance Criteria

At the end of running loadtest.sh, a CSV is generated in the requested format

Including headers would be nice as well.

[kylehuntsman]

I have since updated the way the summary json files are saved. In fact, the new way that makes the file contents easier to distinguish for humans might make this ticket harder. PR can be found here #10

We could trash the entire out directory on subsequent runs. If we do that, then you can basically just read every json file in a test directory without having to know the details of it's name.