rust-lang / rust

Empowering everyone to build reliable and efficient software.
https://www.rust-lang.org
Other
96.69k stars 12.49k forks source link

Bootstrap should skip preparing a linker for targets when checking std #128180

Closed ChrisDenton closed 5 days ago

ChrisDenton commented 1 month ago

Currently ./x check std --stage 0 --target <target> will attempt to configure the target linker and other tools even though they're unused. For the most part this just results in needless cc spam (and presumably slower checks). E.g.:

cargo:warning=Compiler family detection failed due to error: ToolNotFound: Failed to find tool. Is `arm-kmc-eabi-gcc` installed? (see https://github.com/rust-lang/cc-rs#compile-time-requirements for help)
cargo:warning=Compiler family detection failed due to error: ToolNotFound: Failed to find tool. Is `arm-kmc-eabi-g++` installed? (see https://github.com/rust-lang/cc-rs#compile-time-requirements for help)
cargo:warning=Compiler family detection failed due to error: ToolNotFound: Failed to find tool. Is `arm-kmc-eabi-g++` installed? (see https://github.com/rust-lang/cc-rs#compile-time-requirements for help)

But for apple targets this is often a hard failure unless you have the relevant tools. E.g.:

> ./x check std --target aarch64-apple-ios --stage 0
Building bootstrap
    Finished `dev` profile [unoptimized] target(s) in 0.07s

error occurred: Failed to find tool. Is `xcrun` installed? (see https://github.com/rust-lang/cc-rs#compile-time-requirements for help)

Build completed unsuccessfully in 0:00:00

I managed to hack around this issue but I don't know bootstrap code very well so a proper fix would be appreciated.

Hack ```diff diff --git a/src/bootstrap/src/core/builder.rs b/src/bootstrap/src/core/builder.rs index 78fbea2e810..9d3c606aa01 100644 --- a/src/bootstrap/src/core/builder.rs +++ b/src/bootstrap/src/core/builder.rs @@ -2433,7 +2433,9 @@ pub fn new( cmd: &str, // FIXME make this properly typed ) -> Cargo { let mut cargo = builder.cargo(compiler, mode, source_type, target, cmd); - cargo.configure_linker(builder); + if cmd != "check" { + cargo.configure_linker(builder); + } cargo } diff --git a/src/bootstrap/src/utils/cc_detect.rs b/src/bootstrap/src/utils/cc_detect.rs index 20d79e490ec..dcc56de121f 100644 --- a/src/bootstrap/src/utils/cc_detect.rs +++ b/src/bootstrap/src/utils/cc_detect.rs @@ -89,13 +89,11 @@ fn new_cc_build(build: &Build, target: TargetSelection) -> cc::Build { pub fn find(build: &Build) { // For all targets we're going to need a C compiler for building some shims // and such as well as for being a linker for Rust code. - let targets = build - .targets - .iter() - .chain(&build.hosts) - .cloned() - .chain(iter::once(build.build)) - .collect::>(); + let targets: HashSet<_> = if matches!(build.config.cmd, crate::Subcommand::Check { .. }) { + build.hosts.iter().cloned().chain(iter::once(build.build)).collect() + } else { + build.targets.iter().chain(&build.hosts).cloned().chain(iter::once(build.build)).collect() + }; for target in targets.into_iter() { find_target(build, target); } ```
bjorn3 commented 1 month ago

configure_linker is also setting the C/C++ compiler, which is used by the build script of compiler-builtins unless optimized-compiler-builtins is false (and in case of checking rustc, the build script of rustc_llvm uses a C++ compiler). Build scripts do not know if the current build is a check build or regular build.

ChrisDenton commented 1 month ago

Which is why I called my code a "hack". But specifically for checking std without optimized-compiler-builtins (the default for local dev), xcrun is entirely unnecessary.

saethlin commented 1 month ago

configure_linker is also setting the C/C++ compiler, which is used by the build script of compiler-builtins unless optimized-compiler-builtins is false

That seems fine to me, optimized-compiler-builtins is an annoying feature anyway because it requires the LLVM submodule.

and in case of checking rustc, the build script of rustc_llvm uses a C++ compiler

That is also fine, we're not checking the compiler. We're checking std.

What Chris is trying to do here used to work. This issue is a regression in bootstrap. Being able to cross-check is very useful; if/when anyone fixes this we should add cross-checking to CI to make sure this doesn't regress again.

onur-ozkan commented 1 month ago

I managed to hack around this issue but I don't know bootstrap code very well so a proper fix would be appreciated.

Hack

diff --git a/src/bootstrap/src/core/builder.rs b/src/bootstrap/src/core/builder.rs
index 78fbea2e810..9d3c606aa01 100644
--- a/src/bootstrap/src/core/builder.rs
+++ b/src/bootstrap/src/core/builder.rs
@@ -2433,7 +2433,9 @@ pub fn new(
         cmd: &str, // FIXME make this properly typed
     ) -> Cargo {
         let mut cargo = builder.cargo(compiler, mode, source_type, target, cmd);
-        cargo.configure_linker(builder);
+        if cmd != "check" {
+            cargo.configure_linker(builder);
+        }
         cargo
     }

diff --git a/src/bootstrap/src/utils/cc_detect.rs b/src/bootstrap/src/utils/cc_detect.rs
index 20d79e490ec..dcc56de121f 100644
--- a/src/bootstrap/src/utils/cc_detect.rs
+++ b/src/bootstrap/src/utils/cc_detect.rs
@@ -89,13 +89,11 @@ fn new_cc_build(build: &Build, target: TargetSelection) -> cc::Build {
 pub fn find(build: &Build) {
     // For all targets we're going to need a C compiler for building some shims
     // and such as well as for being a linker for Rust code.
-    let targets = build
-        .targets
-        .iter()
-        .chain(&build.hosts)
-        .cloned()
-        .chain(iter::once(build.build))
-        .collect::<HashSet<_>>();
+    let targets: HashSet<_> = if matches!(build.config.cmd, crate::Subcommand::Check { .. }) {
+        build.hosts.iter().cloned().chain(iter::once(build.build)).collect()
+    } else {
+        build.targets.iter().chain(&build.hosts).cloned().chain(iter::once(build.build)).collect()
+    };
     for target in targets.into_iter() {
         find_target(build, target);
     }

These cmd checks in the builder module are going out of control I think. The proper solution would be to refactor Cargo, fix this FIXME, and handle all cmds from a single source (ideally covered by some unit tests). I can allocate some time this weekend to work on that.