search

tinylibs / tinybench

🔎 A simple, tiny and lightweight benchmarking library!
MIT License
1.87k stars 37 forks source link

Per-iteration setup? #32

Closed achingbrain closed 1 year ago

achingbrain commented 1 year ago

Is it possible to do untimed setup before each iteration of a task? I've seen the setup option, but it only gets run before each task that is .added to the benchmark suite rather than each time the task function is executed.

In my benchmarks I need to set up test data each time but I don't want this setup to impact the reported timings.

Aslemammad commented 1 year ago

@achingbrain Hey, Check #31 if this is helpful and send your opinion around it.

achingbrain commented 1 year ago

I saw that but after a bit more consideration it doesn't completely solve the problem as I need control to be a bit more fine-grained.

I'm mostly benchmarking different implementations of the same functionality and need to perform different operations on each implementation to bring them into a similar same state before the task function runs an iteration in order to get useful data out of the benchmark.

I've opened #33 with what solves my problem, hopefully it's acceptable.