BurntSushi / cargo-benchcmp

A small utility to compare Rust micro-benchmarks.
The Unlicense
341 stars 21 forks source link

Ported tool from Python to Rust + added functionality for comparing implementations #2

Closed Apanatshka closed 8 years ago

Apanatshka commented 8 years ago

As per #1, I removed the Python code. Don't forget to change the description of the repo ;)

BurntSushi commented 8 years ago

Thanks @Apanatshka! I've left some comments/nits. If you'd like me to just merge it, I can do that and I'll fix up the code when I get time. :-)

Apanatshka commented 8 years ago

For whatever reason, I can't parse this description. I think I don't know what "name of implementation" means.

Maybe you can help me find a concise description? I don't like the large blurb of text at the start of the usage string anyway.

The idea is that you give the names of two modules that both have the same benchmark tests. The normal scenario would be that the two modules use two different implementations of the same functionality. This is the new example I plan to add to the README:

$ cargo benchcmp dense_boxed dense aho-corasick
name                                dense_boxed ns/iter  dense ns/iter  diff ns/iter  diff %
ac_two_one_prefix_byte_random       21,041               16,741         -4,300        -20.44%
ac_two_one_prefix_byte_no_match     354                  349            -5            -1.41%
ac_two_one_prefix_byte_every_match  150,678              112,962        -37,716       -25.03%
ac_two_diff_prefix                  3,139                3,127          -12           -0.38%
ac_two_bytes                        3,140                3,125          -15           -0.48%
ac_ten_one_prefix_byte_random       23,972               19,495         -4,477        -18.68%
ac_ten_one_prefix_byte_no_match     354                  356            2             0.56%
ac_ten_one_prefix_byte_every_match  150,636              115,112        -35,524       -23.58%
ac_ten_diff_prefix                  108,137              59,237         -48,900       -45.22%
ac_ten_bytes                        108,109              59,331         -48,778       -45.12%
ac_one_prefix_byte_random           20,476               16,515         -3,961        -19.34%
ac_one_prefix_byte_no_match         354                  358            4             1.13%
ac_one_prefix_byte_every_match      150,619              114,608        -36,011       -23.91%
ac_one_byte                         354                  356            2             0.56%
Apanatshka commented 8 years ago

@BurntSushi I think I've addressed all your comments. Can you take another look?

Apanatshka commented 8 years ago

Hmm, I see that the output from the Rust version doesn't do the right justification of the last two columns. I'll look into that.

Apanatshka commented 8 years ago

Definitely don't merge yet, I introduced a problem somewhere that gives: fatal runtime error: out of memory :sweat_smile:

BurntSushi commented 8 years ago

\o/

Apanatshka commented 8 years ago

I dropped the code that was giving trouble because it was useless anyway. As it happens, I accidentally stumbled on a bug in Rust.

I think I'm done with pushing changes. Can you take another look at the code? :)

BurntSushi commented 8 years ago

@Apanatshka Will do. Might take me a little bit. My weekend is packed!

Apanatshka commented 8 years ago

@BurntSushi Do you have any time this weekend? I won't have much time during the week anymore, my summer vacation is over :)

BurntSushi commented 8 years ago

@Apanatshka Thanks for the ping! Let's do this! I'm going to work a little bit with it and once I'm satisfied I'll push it out to crates.io so that folks can get it with cargo install.

Thanks so much for doing this. Really nice work. :-)

Apanatshka commented 8 years ago

Thanks. JSYK: I have more features in the pipeline for this tool

BurntSushi commented 8 years ago

OK, I've done a little bit of refactoring and pushed cargo-benchcmp to crates.io! Thanks again!