search

BurntSushi / cargo-benchcmp

A small utility to compare Rust micro-benchmarks.
The Unlicense
342 stars 21 forks source link

Include results that are not in both result sets #27

Closed sunjay closed 7 years ago

sunjay commented 7 years ago

I have a build using cargo benchcmp. Thanks for making this! It's great!

Currently, my results look like this:

name                   previous-benchmark ns/iter  current-benchmark ns/iter  diff ns/iter  diff % 
b01_compile_trivial    2,132                       2,829                               697  32.69% 
b02_compile_large      1,594,916                   2,516,662                       921,746  57.79% 
b03_compile_huge       51,480,362                  86,575,722                   35,095,360  68.17% 
b04_compile_simple     31,399                      44,247                           12,848  40.92% 
b05_compile_slow       225,000                     289,571                          64,571  28.70% 
b06_interpret_trivial  10,325                      10,442                              117   1.13% 
b07_interpret_simple   6,740,466                   7,388,517                       648,051   9.61% 
WARNING: benchmarks in new but not in old: b01_compile_trivial_opt, b02_compile_large_opt, b03_compile_huge_opt, b04_compile_simple_opt, b05_compile_slow_opt, b06_interpret_trivial_opt, b07_interpret_simple_opt

The WARNING on the bottom is because I added a bunch of benchmarks in the pull request I'm looking at. It would be great if instead of the warning, benchcmp listed those benchmarks as part of the results. Just leave the other column blank and don't color the text (leave it white or the default). That way all the results will show up in the table, and I can still clearly see what changed from before to after.

BurntSushi commented 7 years ago

I'd be fine with that change. I doubt I'll get to it myself any time soon, but a PR would be welcome.

sunjay commented 7 years ago

@BurntSushi Here you go! https://github.com/BurntSushi/cargo-benchcmp/pull/28