What: an option like assert_all_same: true which would run all functions (for each input if using inputs) and ensure that results are equal for each input across functions.
Context: I've been mostly using Benchee to optimize pure functions in core, which means benchmarking alternative implementations. Failing early if one of my alternative implementation is not equivalent would improve the feedback loop, since I'm usually not writing tests yet during the benchmarking phase (but I often end up adding some dbg or manual checks).
I could write my own small wrapper, but since it might be a common use case I went for suggesting it in Benchee itself.
Hi @PragTob, how are you doing?
What: an option like
assert_all_same: truewhich would run all functions (for each input if usinginputs) and ensure that results are equal for each input across functions.Context: I've been mostly using Benchee to optimize pure functions in core, which means benchmarking alternative implementations. Failing early if one of my alternative implementation is not equivalent would improve the feedback loop, since I'm usually not writing tests yet during the benchmarking phase (but I often end up adding some
dbgor manual checks).I could write my own small wrapper, but since it might be a common use case I went for suggesting it in Benchee itself.
I'd be happy to send a PR if there is interest.