While using perf:library I found that I wanted to know what directory a given test run was stored in. So now the output looks like this:
👎👎👎(NOT Statistically Significant) 👎👎👎
[3054e1d584] "Merge pull request #36506 from kamipo/group_by_with_order_by_virtual_count_attribute" - (0.017138 seconds)
SLOWER by:
0.9702x [older/newer]
-3.0764% [(older - newer) / older * 100]
[80f989aece] "Remove duplicated attribute alias resolution in `_select!`" - (0.0166265 seconds)
Iterations per sample: 10
Samples: 2
Test type: Kolmogorov Smirnov
Is significant? (max > critical): false
D max: 0.5
D critical: 1.2238734153404083
Output: tmp/compare_branches/2019-12-16-21-20-1576552858-200576000/results.txt
Note the file in the bottom which has the directory name tmp/compare_branches/2019-12-16-21-20-1576552858-200576000
I also found that in addition to the raw results I wanted the calculations (which was faster, by how much, and was it statistically significant or not). So in addition to printing the banner to the output, i'm now also writing it to the results file which you see in the output string.
In the process of adding these features, i found tests were failing
$ BUNDLE_GEMFILE="$(pwd)/gemfiles/rails_git.gemfile" bundle exec m test/integration/tasks_test.rb:30
Which showed that the issue is stats.banner.call should have been stats.call.banner so I think this PR fixes the test suite.
While using
perf:libraryI found that I wanted to know what directory a given test run was stored in. So now the output looks like this:I also found that in addition to the raw results I wanted the calculations (which was faster, by how much, and was it statistically significant or not). So in addition to printing the banner to the output, i'm now also writing it to the
resultsfile which you see in the output string.In the process of adding these features, i found tests were failing
Which showed that the issue is
stats.banner.callshould have beenstats.call.bannerso I think this PR fixes the test suite.