search

nikomatsakis / rust-runtime-benchmarks

Some benchmarks for testing rustc performance.
8 stars 2 forks source link

Create a runner #2

Open nikomatsakis opened 8 years ago

nikomatsakis commented 8 years ago

Right now you have to run cargo bench manually. This is not good. We should have a runner that goes into each directory and runs it for you and then outputs the times from each time into a file. We should then a way to compare two files and show the change for each test (i.e., time before, time after, speedup percentage).

Ideally this same tool would also run with -Z time-passes and save the time from compilation, which could then be compared and graphed and whatever'd as well.

BurntSushi commented 8 years ago

@nikomatsakis You may find cargo-benchcmp useful for comparing the output of two runs of cargo bench.

anp commented 8 years ago

I had an overly ambitious desire to make a benchmark runner: https://github.com/dikaiosune/secondstring. I've since been distracted by other things, but I think it could potentially be adapted to cover custom toolchains as well by switching to rustup. If there's interest I'll take some time later this week to see if I can adapt it to the benchmarks here.