search

rust-lang / crater

Run experiments across parts of the Rust ecosystem!
https://crater.rust-lang.org
642 stars 90 forks source link

Consider checking tests in check-only mode #549

Open Aaron1011 opened 4 years ago

Aaron1011 commented 4 years ago

Currently, we only check crates in the workspace for a check-only build:

https://github.com/rust-lang/crater/blob/dcc54566d5ec82cabca435c0145cbd06f32d3602/src/runner/test.rs#L302-L308

However, we do not pass --tests, which can cause us to miss some regressions. See https://github.com/rust-lang/rust/issues/77789#issuecomment-706600489

ehuss commented 3 years ago

The --all-targets flag causes it to check all tests.

I don't think crater missed the regression noted in the link. When crater was ran, try_match 0.2.2 was using these dependencies:

proc-macro2 v1.0.18 version_check v0.9.2 unicode-xid v0.2.0 syn v1.0.31 proc-macro-hack v0.5.16 proc-macro-error-attr v1.0.2 proc-macro-error v1.0.2 quote v1.0.7 syn-mid v0.5.0 try_match_inner v0.2.0

With that set, it builds fine. It's only when you update (I'm not sure which) that the regression happens. So it was just a timing issue.