Support for micro benchmarks on data structures

[GoogleCodeExporter]

What steps will reproduce the problem?
1. I tried to do micro benchmarks on data structures (lists, sets, etc)
2. Operations such as "add" and "remove" are very hard to measure without a 
certain control over the reps. On "add" you may end up running out of memory. 
On "remove" you may end up with a NullPointerException if the data structure 
does not have enough items stored.

What is the expected output? What do you see instead?
What helps is to have the @BeforeRep annoations to rebase the data structure 
for every new measurement round. This avoid heap memory overflow.
Second, it helps to have a minimum of control on the "reps" parameter such that 
the NPE for the "remove" experiments are being avoided.

So I enhanced the RuntimeWorker.Micro with the code from RuntimeWorker.Macro to 
support the @BeforeRep/@AfterRep methods for rebasing. Finally, the @BeforeRep 
takes the calculated number of reps as an input parameter and return the max 
number of reps.

In case you find these additions interesting, the code looks like so:

  public static final class Micro extends RuntimeWorker {
    private final ImmutableSet<Method> beforeRepMethods;
    private final ImmutableSet<Method> afterRepMethods;

    @Inject Micro(@Benchmark Object benchmark, 
        @BenchmarkMethod Method method, Random random, Ticker ticker,
        @WorkerOptions Map<String, String> workerOptions) {
      super(benchmark, method, random, ticker, workerOptions);
      this.beforeRepMethods =
          getAnnotatedMethods(benchmark.getClass(), BeforeRep.class);
      this.afterRepMethods =
          getAnnotatedMethods(benchmark.getClass(), AfterRep.class);
    }

    @Override public void preMeasure() throws Exception {
      super.preMeasure();
      for (Method beforeRepMethod : beforeRepMethods) {
        Object result = beforeRepMethod.invoke(benchmark, super.nextReps);
        if(result != null && result instanceof Integer){
          super.nextReps = Math.min(super.nextReps, (Integer) result);   
        }
      }
    }

    @Override long invokeTimeMethod(long reps) throws Exception {
      int intReps = (int) reps;
      if (reps != intReps) {
        throw new InvalidBenchmarkException("%s.%s takes an int for reps, "
            + "but requires a greater number to fill the given timing interval (%s). "
            + "If this is expected (the benchmarked code is very fast), use a long parameter."
            + "Otherwise, check your benchmark for errors.",
                benchmark.getClass(), benchmarkMethod.getName(),
                    ShortDuration.of(options.timingIntervalNanos, NANOSECONDS));
      }
      long before = ticker.read();
      benchmarkMethod.invoke(benchmark, intReps);
      return ticker.read() - before;
    }

    @Override public void postMeasure() throws Exception {
      super.postMeasure();
      for (Method afterRepMethod : afterRepMethods) {
        afterRepMethod.invoke(benchmark, super.nextReps);
      }
    }
  }

What version of the product are you using? On what operating system?
caliper-1.0-beta

Please provide any additional information below.

Original issue reported on code.google.com by sebastia...@gmail.com on 28 Jul 2014 at 8:50