search

ltratt / multitime

Time command execution over multiple executions
http://tratt.net/laurie/src/multitime/
MIT License
115 stars 13 forks source link

Report hypothesis tests in batch mode #6

Open snim2 opened 9 years ago

snim2 commented 9 years ago

In batch mode multitime can produce timing measurements for a number of different commands. One obvious use of this is to compare the performance of two similar commands. In this situation, it would be useful to report the result of a Student t-test (or similar hypothesis test) which compares the sample means of the different commands.

ltratt commented 9 years ago

I must admit to not really understanding the student t-test off hand, but something like this sounds sensible.

snim2 commented 9 years ago

OK, I'll work out the statistics and come up with a more detailed proposal.

snim2 commented 9 years ago

DECISION: Do this anyhow (!) but put the code in extras/