search

felixge / sqlbench

sqlbench measures and compares the execution time of one or more SQL queries.
MIT License
368 stars 11 forks source link

Baseline #3

Closed felixge closed 4 years ago

felixge commented 4 years ago

Support benchmarking against a recorded baseline. Useful when comparing different PostgreSQL versions, server configurations, indexes, etc.

Hopefully this will help with the use cases suggested by @anarazel on twitter: https://twitter.com/AndresFreundTec/status/1308146736917962752

# realtime output, and record the results for 1000 iterations into a csv file
sqlbench -n 1000 -o baseline.csv examples/sum/*.sql
# realtime output, and compare 1000 iterations to a baseline recording
sqlbench -n 1000 -i baseline.csv examples/sum/*.sql

Output from second run against the baseline (same PG version and settings, so no big diffs expected)

         |    gauss     |    window    |  recursive    
---------+--------------+--------------+---------------
  n      | 1000 (1.00x) | 1000 (1.00x) | 1000 (1.00x)  
  min    | 0.35 (0.99x) | 1.24 (0.98x) | 1.81 (0.98x)  
  max    | 0.80 (1.00x) | 2.20 (0.91x) | 3.26 (0.99x)  
  mean   | 0.39 (1.00x) | 1.36 (0.99x) | 2.00 (0.99x)  
  stddev | 0.04 (0.76x) | 0.09 (0.76x) | 0.14 (0.86x)  
  median | 0.37 (1.01x) | 1.32 (0.99x) | 1.95 (0.99x)  
  p90    | 0.42 (0.96x) | 1.47 (0.98x) | 2.17 (0.98x)  
  p95    | 0.45 (0.95x) | 1.54 (0.96x) | 2.28 (0.97x)
felixge commented 4 years ago

Merged in 4c97321915ce8e971853a0df3f08d6c71bd42397