ordo-one / package-benchmark

Swift benchmark runner with many performance metrics and great CI support
Apache License 2.0
313 stars 25 forks source link

Any API for specifying `minIterations` on a benchmark? #238

Closed vanvoorden closed 5 months ago

vanvoorden commented 6 months ago

Hi! I'm running some benchmarks that I want to guarantee to run for n iterations without timing out. I'm trying this:

Benchmark.defaultConfiguration.maxDuration = SOME_LARGE_DURATION
Benchmark.defaultConfiguration.maxIterations = .count(n)

This seems to work… but is there any alternative API for me to use for that? Something like minIterations or a DISTANT_FUTURE flag that can be passed for a duration?

hassila commented 6 months ago

Hi,

That is indeed the intended approach - usually even if you want to run N iterations you probably want to timeout at some point especially for eg. CI - but we haven't added any convenience wrapper to duration for that (yet)

vanvoorden commented 6 months ago

@hassila Thanks!