lorenzwalthert / touchstone

Smart benchmarking of pull requests with statistical confidence
https://lorenzwalthert.github.io/touchstone
Other
53 stars 7 forks source link

Don't make `bench::mark` check results. #139

Closed plietar closed 2 months ago

plietar commented 2 months ago

By default, the bench::mark function used by touchstone runs one additional iteration of the benchmark, whose result it keeps and uses to check for equivalence in behaviour of the given expressions.

This feature doesn't make sense in the context of touchstone, where only one version is being benchmarked at a time. Moreover, given that bench::mark is always called to run a single iteration, this behaviour effectively doubles the number of iterations being run but without keeping the results from these extra runs, which was wastefuly doubling the time taking to run the action.

We can avoid this behaviour by passing check = FALSE as an argument to the function.