Option to regress on any output change

rust-lang / cargo-bisect-rustc

Bisects rustc, either nightlies or CI artifacts

https://rust-lang.github.io/cargo-bisect-rustc/

Apache License 2.0

177 stars 55 forks source link

Option to regress on any output change #139

Open JakobDegen opened 2 years ago

JakobDegen commented 2 years ago

Some more complicated situations do not allow one to find an example that cleanly succeeds/fails depending on rustc version. Especially when hacking on rustc is eg find an example that produces one error more/less than it should. Adding support for this situation by having an output that just reports any change as a regression would be useful.

langston-barrett commented 1 year ago

Is this something that can be accomplished with "Testing with a script"? https://github.com/rust-lang/cargo-bisect-rustc/blob/95eaf5b24e00efa5ce5415ebdd6c8814dfc80720/TUTORIAL.md#testing-with-a-script