Saved Results Feature Request

[GoogleCodeExporter]

Caliper should allow you to save results into source control and support 
comparisons of your trunk code against a historic version.

This would be useful when optimising the performance of some code as you can 
easily confirm the change has made a noticeable improvement.

Original issue reported on code.google.com by yuri.sch...@gmail.com on 24 Feb 2011 at 9:02

[GoogleCodeExporter]

I'm not sure I understand your full meaning.  Technically, you _can_ save 
results into source control, just use --saveResults and check the file in. 
That's probably not what you meant. :)

Without really understanding, I'll say that this *sounds* like the kind of cool 
feature that is meant to be built up *around* caliper.  Caliper's purpose is 
focused on the taking of measurements, and the storing and basic viewing of 
measurements.

Original comment by kevinb@google.com on 24 Feb 2011 at 3:10

[GoogleCodeExporter]

Agreed, it probably makes sense to have this as a add-in.  However if that is 
the case I would hope plugins could be involved in the normal runners.

What I would hope could be achieved would be something like

Benchmark           ns      compared to baseline
-----------------  ---  ----------------
NanoTime           248    -20%
CurrentTimeMillis  118    -38%

Original comment by yuri.sch...@gmail.com on 27 Feb 2011 at 7:42

[GoogleCodeExporter]

I guess the other feature this suggests is failing the benchmark if results 
have regressed by some amount e.g. +20%.

This would obviously require the the same host machine.

Original comment by yuri.sch...@gmail.com on 27 Feb 2011 at 7:44

[GoogleCodeExporter]

I'm gong to declare this out of scope.  First let's get the process of taking 
the measurements nailed.

Original comment by kevinb@google.com on 19 Mar 2011 at 2:18

• Changed state: WontFix