search

netguy204 / cljs-bench

Runs benchmarks against a range of Clojurescript versions and reports the results
http://www.50ply.com/cljs-bench/
10 stars 2 forks source link

Less noisy graphs #1

Open timmc opened 12 years ago

timmc commented 12 years ago

The graphs would be easier to interpret if the y-axis were anchored to zero.

A minimum range of about 3x the height of the noise floor would also be nice.

(Sorry, I don't know gnuplot syntax, otherwise I'd send a pull request.)

netguy204 commented 12 years ago

Now anchored to zero.

Do you think that the standard deviation be a reasonable surrogate for computing the noise-floor or did you have a specific approach in mind.

timmc commented 12 years ago

Hmm, now the tops of some of the graphs are getting cut off. :-/

By noise-floor I suppose I meant the granularity of your benchmarking tool (as an acceptable surrogate.) If you can't reliably measure with greater precision than (say) 10 ms, then maybe the graph should be at least 30 ms tall.

netguy204 commented 12 years ago

I see. I don't actually have enough data collected yet to estimate that value.

In the mean time, a range of 0 through (max 300 (min highest-point (+ lowest-point (* 3 sigma))))

looks pretty good (to me.)

On Tue, May 29, 2012 at 9:52 PM, Tim McCormack < reply@reply.github.com

wrote:

Hmm, now the tops of some of the graphs are getting cut off. :-/

By noise-floor I suppose I meant the granularity of your benchmarking tool (as an acceptable surrogate.) If you can't reliably measure with greater precision than (say) 10 ms, then maybe the graph should be at least 30 ms tall.

Reply to this email directly or view it on GitHub: https://github.com/netguy204/cljs-bench/issues/1#issuecomment-6000511