[Enhancement] Benchmarking JUnit tests

[GoogleCodeExporter]

I think it would be great to have an integration with JUnit.
I'm looking forward to add @Benchmark to a JUnit test class and have all of the 
@Test methods benchmarked, as well as integrating JUnit's @Before and @After 
with @BeforeExperiment and @AfterExperiment.
Maybe an adapter for JUnit Runner?
I think it could give Caliper a boost. The Java community would love this.
What do you think?

Cheers.

Original issue reported on code.google.com by mariano....@gmail.com on 17 Sep 2013 at 5:26

[GoogleCodeExporter]

I have a few concerns about trying to benchmark test methods:
1) Junit's @Test methods are incompatible with microbenchmarks due to the int 
param
2) I'm not sure whether or not @Test methods will complain if there is a return 
type.
3) There could be non-trivial overhead for any of the assertions being run in a 
test method. E.g.: string matching.  It could be quite easy to end up 
benchmarking the assertions rather than the code being tested.

On the other hand, running @Benchmark methods as tests is something we do quite 
often, but without any proper integration.  Making sure that benchmarks don't 
decay over time is important and the ability to annotate a benchmark class with 
@RunWith(CaliperDryRun.class) or something similar would be useful.

Original comment by gak@google.com on 17 Sep 2013 at 5:25