Add JSON output type to grandma diff

catdad / grandma

👵 fully programmable stress testing framework

20 stars 4 forks source link

Add JSON output type to grandma diff #198

Open gsf4726 opened 4 years ago

gsf4726 commented 4 years ago

Currently grandma.diff output only supports tabular text like:

Latencies:  mean               50                 95                 99                 max

run: someTest, 10s, 1 concurrent, 3 total
fullTest    4650.651ms (-35%)  4686.128ms (-33%)  4880.873ms (-33%)  4880.873ms (-33%)  4880.873ms (-33%)

bench: someTest, 10s, 1 concurrent, 3 total
fullTest    3456.642ms         3527.152ms         3674.297ms         3674.297ms         3674.297ms

Adding a JSON output option would make it easier for an automated process to handle the results.

catdad commented 4 years ago

Hmm... what info are you interested in having in a JSON diff? You can already get a JSON for individual runs. What would a diff JSON look like?

gsf4726 commented 4 years ago

Was primarily thinking, to implement a failure threshold in automation, it'd be convenient to get the deltas that are currently reported in the diff text output.

Certainly a convenience though.