search

rust-lang / crater

Run experiments across parts of the Rust ecosystem!
https://crater.rust-lang.org
629 stars 88 forks source link

Add more exceptions to the list #671

Closed est31 closed 1 year ago

est31 commented 1 year ago

I would also love #667 to be merged, but this probably has to happen after this PR as this PR fixes clippy.

est31 commented 1 year ago

r? @Mark-Simulacrum

rustbot commented 1 year ago

Error: This repository is not enabled to use triagebot. Add a triagebot.toml in the root of the default branch to enable it.

Please file an issue on GitHub at triagebot if there's a problem with this bot, or reach out on #t-infra on Zulip.

est31 commented 1 year ago

Oh this needs to wait until after https://github.com/rust-lang/rust/pull/103654 has been merged and put into nightly (edit: or I can just address the lint anyways).

Mark-Simulacrum commented 1 year ago

@bors r+

bors commented 1 year ago

:pushpin: Commit 02280c654abe0cb092324f2708209423704bd3ee has been approved by Mark-Simulacrum

It is now in the queue for this repository.

bors commented 1 year ago

:hourglass: Testing commit 02280c654abe0cb092324f2708209423704bd3ee with merge 53db254d5729b31a657c148a69fa6fbbf8d537e8...

bors commented 1 year ago

:sunny: Test successful - checks-actions Approved by: Mark-Simulacrum Pushing 53db254d5729b31a657c148a69fa6fbbf8d537e8 to master...

RalfJung commented 1 year ago

FWIW these are probably almost all caused by https://github.com/rust-lang/crater/issues/663. Basically every crate that links libc or a proc macro shows these symptoms. Excluding some of them is fairly pointless unless we aim at eventually excluding all of them... if we have 10k crates that fail with a rate of 0.1%, excluding 10 of them makes no difference whatsoever.

est31 commented 1 year ago

I agree that it's pointless to add crates that "won" the 0.1% lottery. And I did indeed encounter a lot of instances of #663, but the exceptions this PR is adding are different. Unlike #663, these failures were consistent: the same crates showed up in an earlier crater report too of an unrelated PR, see the commit msg of 573d5df568a0acf54602279ccc2214b32f586973 .

I'm not 100% sure how these errors occured but they seem to be consistent. For binja-rs, I have somewhat of an idea: OUT_DIR does not change between nightly versions or different rustc builds (not sure why but local testing confirms that), and both the new and old build.rs are putting a symlink into OUT_DIR. Why the check "does the symlink already exist" doesn't work I'm not sure, maybe due to the race condition that the Path::exists docs warn you about? IDK. For the snippet of the relevant code, see the "file already exists error" section of my triage comment, but note that I was wrong with the path being erroneously calculated, upon second inspection it was calculated correctly. The glsl validation issue, which occurs in a proc macro, is less clear to me how it comes to be... but it does also fail consistently.

est31 commented 1 year ago

FTR, except for vulkano_tri1 and vulkano_examples, I could also find the things this PR is adding in both crater runs of your PR: https://github.com/rust-lang/rust/pull/102513#issuecomment-1264515654 and https://github.com/rust-lang/rust/pull/102513#issuecomment-1264568966 . I've checked the vulkano repos, they seem to run into OOM issues (listed as build-fail), probably due to an artifact of the build environment, so likely that's why the bug is not surfacing.

Note that I sided on the side of caution when making this PR, and thus this PR only added a tiny subset of the spurious failures to the blacklist.

RalfJung commented 1 year ago

Ah, fair. :)

The symlimk part does not make sense to me though, doesn't crater run baseline and try build in completely separate containers?

est31 commented 1 year ago

doesn't crater run baseline and try build in completely separate containers?

If you compare the before and after (i open them in two tabs, scroll to the same position, and then switch back and forth using ctrl + pg up / down), you see that the it downloads some dependencies both from github as well as from crates.io in one job.

Also if you check the invoked commands, it seems to set CARGO_TARGET_DIR=/opt/rustwide/target in the docker container, and also call the -v flag to mount a directory from the host into the container:

/var/lib/crater-agent-workspace/builds/worker-7/target:/opt/rustwide/target

Maybe that mapping also explains why the check is going wrong: the symlink is seen as non-existent by the check because it doesn't resolve in the host's file system.