rust-lang / crater

Run experiments across parts of the Rust ecosystem!
https://crater.rust-lang.org
627 stars 88 forks source link

Automatic detection of spurious regressions #701

Open RalfJung opened 10 months ago

RalfJung commented 10 months ago

Would it make sense to run crater on two identical commits every now and then (when the queue is empty), and then add all crates that regress on that run to the "spurious" list? Clearly those regressions have to be spurious since the compiler didn't actually change.