zombocom / derailed_benchmarks

Go faster, off the Rails - Benchmarks for your whole Rails app
2.96k stars 144 forks source link

Add commit description and ordering to output #139

Closed schneems closed 5 years ago

schneems commented 5 years ago

There were some problems to the old formatting of the library comparison output. The x faster and % faster were confusing because it wasn't clear how they were derived. Also it was not clear which commit SHA belonged to which values.

In this updated value we give the formulas for the values that are being shown as well. The latest commit is always first and the value will show either SLOWER or FASTER based on actual values. Finally we're including the git first line description to make it clearer which commit is which.

New output looks like this:

❤️ ❤️ ❤️  (Statistically Significant) ❤️ ❤️ ❤️

[7b4d80cb37] "1.8x Faster Partial Caching - Faster Cache Keys" - (10.9711965 seconds)
  FASTER by:
    1.0870x [older/newer]
    8.0026% [(older - newer) / older * 100]
[13d6aa3a7b] "Merge pull request #36284 from kamipo/fix_eager_loading_with_string_joins" - (11.9255485 seconds)

P-value: 4.635595463712749e-05
Is significant? (P-value < 0.05): true