Open GoogleCodeExporter opened 9 years ago
I have a few concerns about trying to benchmark test methods:
1) Junit's @Test methods are incompatible with microbenchmarks due to the int
param
2) I'm not sure whether or not @Test methods will complain if there is a return
type.
3) There could be non-trivial overhead for any of the assertions being run in a
test method. E.g.: string matching. It could be quite easy to end up
benchmarking the assertions rather than the code being tested.
On the other hand, running @Benchmark methods as tests is something we do quite
often, but without any proper integration. Making sure that benchmarks don't
decay over time is important and the ability to annotate a benchmark class with
@RunWith(CaliperDryRun.class) or something similar would be useful.
Original comment by gak@google.com
on 17 Sep 2013 at 5:25
Original issue reported on code.google.com by
mariano....@gmail.com
on 17 Sep 2013 at 5:26