search

JuliaCI / BenchmarkTools.jl

A benchmarking framework for the Julia language
Other
604 stars 97 forks source link

timeout measures warmup (eliminating many of the reasons to use timeout) #368

Closed willow-ahrens closed 2 months ago

willow-ahrens commented 2 months ago

The issue is with these lines here: https://github.com/JuliaCI/BenchmarkTools.jl/blob/0941fd1c458fff6db6a90d4ca098f38381bed93f/src/execution.jl#L113-L117

The problem is that we time out if the warmup takes a long time, because we start the clock before the warmup

willow-ahrens commented 2 months ago

Closed in https://github.com/JuliaCI/BenchmarkTools.jl/pull/367