ltratt / multitime

Time command execution over multiple executions
http://tratt.net/laurie/src/multitime/
MIT License
115 stars 13 forks source link

Improve reporting of mean times + CI #3

Open snim2 opened 9 years ago

snim2 commented 9 years ago

Here is an example run of multitime:

===> multitime results
1: ./demosaic images/airplane_RGGB.png
             Mean                Std.Dev.    Min         Median      Max
real        1.563+/-0.3564      0.159       1.444       1.512       2.177       
user        1.408+/-0.0019      0.012       1.385       1.407       1.435    
sys         0.024+/-0.0012      0.009       0.012       0.024       0.052 

In this example, the wall clock time is not useful, but the amount of time spent on the CPU is of interest. Unfortunately, summing the user and sys rows is no longer valid, because confidence intervals are not additive.

Therefore, I propose a new list of rows to display on the output:

ltratt commented 9 years ago

I'm open to seeing 'real' renamed to 'wall' (except in time compatibility mode when it must remain as 'real' for scripts that expect to scrape time's output).

However, I'm not sure that there's ever been a guarantee that real=user+sys, even in the traditional Unix time program? Perhaps because of that, I'm a bit nervous about suggesting that CPU time is just a factor of those two, because there might be other factors that contribute to CPU time that one simply can't get measurements for.

snim2 commented 9 years ago

Perhaps I have misunderstood time. I thought 'real' was the wall clock time, 'user' was the time spent on the CPU in user mode, and 'sys' was the time spent on the CPU in kernel mode.

So, in my proposed format 'wall' is the wall clock time, 'cpu' is 'user' + 'sys' which we would expect to be less than 'wall' (if the process has spent time waiting) or greater than 'wall' (if the process has parallel tasks in a multicore system).

It seems that adding 'user' and 'sys' is quite a common operation. Perhaps there is a better name for it than 'cpu', but that is how I have seen it referred to.

snim2 commented 9 years ago

I pushed a first go at this to a branch called "cputime".

ltratt commented 9 years ago

Do you want to make this a pull request?

snim2 commented 9 years ago

OK, done.