search

rust-lang / cargo

The Rust package manager
https://doc.rust-lang.org/cargo
Apache License 2.0
12.54k stars 2.38k forks source link

Migrate away from ad-hoc snapshot testing for rustfix test suite #13891

Open weihanglo opened 4 months ago

weihanglo commented 4 months ago

Problem

When dealing with rustfix snapshot test fixes, I found it a bit cumbersome to figure out the actual failure. You need to pass RUST_LOG=parse_and_replace=info to see the log, and RUSTFIX_TEST_BLESS=test-name.rs to update snapshots.

Proposed Solution

In the main Cargo crate, we've already integrated snapbox for UI tests. We could migrate rustfix test suite to that, and maybe share some common infra, like nightly channel detection from cargo-test-macro.

Notes

ehuss has a proposal that we don't run rustc if JSON snapshot exists: https://github.com/rust-lang/cargo/pull/13890#issuecomment-2102747898. This could be a good enhancement, and I think we can leave it to follow-ups.

yatinmaan commented 1 month ago

@rustbot claim

epage commented 1 month ago

FYI I had looked into this in the past and it was a bit messy because we only care about updating the rustc-output snapshots if the rustfix snapshots fail.