search

google / caliper

Micro-benchmarking library for Java
Apache License 2.0
809 stars 109 forks source link

Say something about (a) JMH and (b) the Jetpack Benchmark library #453

Closed cpovirk closed 1 year ago

cpovirk commented 3 years ago

My understanding is that our recommendation is that people should use JMH. We make an exception for Android (where JMH doesn't work), but I don't think Caliper works under Android externally, so we wouldn't need to cover that here.

(I recall discussing years ago how it would be nice if we could import JMH data into Caliper's pivot-table UI, but it's clear that's not happening anytime soon.)

harshita19244 commented 3 years ago

@cpovirk could you please specify why JMH is recommended over Caliper? I am analyzing various tools available for writing benchmark tests, and it would be good to know why should I use JMH over Caliper.

cpovirk commented 3 years ago

The best explanation I have handy is https://groups.google.com/g/mechanical-sympathy/c/m4opvy4xq3U/m/7lY8x8SvHgwJ?pli=1

harshita19244 commented 3 years ago

Is it still relevant, as most comments are from 6yrs ago

cpovirk commented 3 years ago

I don't think we have addressed any of the issues discussed there in the last ~4~ 6 years. If anything, JMH has probably widened its lead over Caliper in that time. (Our focus has been on Android, where we can't use JMH, and on general maintenance.)

(In everything I'm saying here, I'm focused on benchmarks of runtime. I don't know what JMH does for allocation benchmarks.)

cpovirk commented 3 years ago

Sorry, typo: I don't think we have addressed any of those issues in the last 6 years.

harshita19244 commented 3 years ago

Thank you so much, this was really helpful

cpovirk commented 2 years ago

A Caliper alternative on Android is the Jetpack Benchmark library (via). We should say something about that, too.