rust-lang / cargo-bisect-rustc

Bisects rustc, either nightlies or CI artifacts
https://rust-lang.github.io/cargo-bisect-rustc/
Apache License 2.0
174 stars 55 forks source link

take increasingly bigger steps #276

Open matthiaskrgr opened 1 year ago

matthiaskrgr commented 1 year ago

After some time, bisect will always check 2 rust versions per month as it goes back in time, but when bisecting stable-to-stable regressions, this still takes a lot of time. It would be nice if could perhaps try to go back by one month at a time a year after the first checked commit and maybe 1.5 months after 2 years? (or just increase the number of days skipped linearly with the further back we go?)

oli-obk commented 1 year ago

we could also just go back exponentially by default and then do a normal bisection when we hit a change

Mark-Simulacrum commented 1 year ago

The choice here was intentional, in my experience the farther back you go the more likely it is that you stop building for different reasons (typically dropping back before the relevant feature is supported), meaning that the "did it build" function is not monotonic, which means you can't bisect: you have bad commits both at the beginning and the end of your curve, which prevents the typical binary search from working.

We can tweak the exact constant, but I would avoid trying anything exponential - that goes too quickly, in my experience.

matthiaskrgr commented 1 year ago

Could be behind an extra cmdline flag. From doing a couple bisections today i already got around 100 bisector toolchains installed which seems like quite a lot .