rust-lang / crater

Run experiments across parts of the Rust ecosystem!
https://crater.rust-lang.org
642 stars 90 forks source link

Automatic detection of spurious regressions #701

Open RalfJung opened 1 year ago

RalfJung commented 1 year ago

Would it make sense to run crater on two identical commits every now and then (when the queue is empty), and then add all crates that regress on that run to the "spurious" list? Clearly those regressions have to be spurious since the compiler didn't actually change.