rust-lang / wg-cargo-std-aware

Repo for working on "std aware cargo"
133 stars 8 forks source link

It is not possible to use `-Zbuild_std` with a Development-Channel Rust #58

Closed cr1901 closed 3 years ago

cr1901 commented 3 years ago

My Custom Rust

I have compiled my own Rust (commit 09f4c9f5082) in an attempt to test an LLVM patch for MSP430:

william@xubuntu-dtrain:~/Projects/embedded/msp430/AT2XT$ rustup toolchain list -v
nightly-x86_64-unknown-linux-gnu (default)      /home/william/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu
msp430-fix (override)   /home/william/Projects/embedded/msp430/AT2XT/../../../toolchains/rust-msp430-fix/build-rust/build/x86_64-unknown-linux-gnu/stage1
william@xubuntu-dtrain:~/Projects/embedded/msp430/AT2XT$ rustc -V
rustc 1.47.0-dev

This Rust compiler's source directory symlinks directly to my git checkout of the rust source:

william@xubuntu-dtrain:~/Projects/embedded/msp430/AT2XT$ ls -l /home/william/Projects/toolchains/rust-msp430-fix/build-rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/src/
total 0
lrwxrwxrwx 1 william william 38 Aug  8 01:09 rust -> /home/william/Projects/toolchains/rust

Attempt At Installing rust-src

When I attempt to install rust-src via rustup, as recommended by the manual, I get the following error telling me exactly what I need is impossible:

rustup component add rust-src --toolchain msp430-fix
error: msp430-fix is a custom toolchain

Okay, fair enough. Let's see what happens when I try building anyway:

cargo build -Zbuild-std=core --release --target=msp430-none-elf
error: failed to read `/home/william/Projects/toolchains/rust-msp430-fix/build-rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/src/rust/src/libstd/Cargo.toml`

Caused by:
  No such file or directory (os error 2)
error: Recipe `timer` failed on line 10 with exit code 10

Indeed, in my msp430-fix toolchain, the libstd doesn't exist, while in my nightly toolchain, the relevant directory does exist:

william@xubuntu-dtrain:~/Projects/embedded/msp430/AT2XT$ ls -l /home/william/Projects/toolchains/rust-msp430-fix/build-rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/src/rust/src/libstd
ls: cannot access '/home/william/Projects/toolchains/rust-msp430-fix/build-rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/src/rust/src/libstd': No such file or directory
william@xubuntu-dtrain:~/Projects/embedded/msp430/AT2XT$ ls -l /home/william/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/
total 852
-rw-r--r--  1 william william  13461 Aug  8 01:57 alloc.rs
-rw-r--r--  1 william william   7117 Aug  8 01:57 ascii.rs
-rw-r--r--  1 william william  18356 Aug  8 01:57 backtrace.rs
drwxrwxr-x  3 william william   4096 Aug  8 01:57 benches
-rw-r--r--  1 william william   4039 Aug  8 01:57 build.rs
-rw-r--r--  1 william william   2802 Aug  8 01:57 Cargo.toml
drwxrwxr-x  3 william william   4096 Aug  8 01:57 collections
-rw-r--r--  1 william william  32341 Aug  8 01:57 env.rs
-rw-r--r--  1 william william  23938 Aug  8 01:57 error.rs
-rw-r--r--  1 william william  58155 Aug  8 01:57 f32.rs
-rw-r--r--  1 william william  59056 Aug  8 01:57 f64.rs
drwxrwxr-x  2 william william   4096 Aug  8 01:57 ffi
-rw-r--r--  1 william william 120858 Aug  8 01:57 fs.rs
-rw-r--r--  1 william william    495 Aug  8 01:57 future.rs
drwxrwxr-x  2 william william   4096 Aug  8 01:57 io
-rw-r--r--  1 william william  65449 Aug  8 01:57 keyword_docs.rs
-rw-r--r--  1 william william  24660 Aug  8 01:57 lazy.rs
-rw-r--r--  1 william william  21118 Aug  8 01:57 lib.rs
-rw-r--r--  1 william william   9199 Aug  8 01:57 macros.rs
-rw-r--r--  1 william william   3187 Aug  8 01:57 memchr.rs
drwxrwxr-x  2 william william   4096 Aug  8 01:57 net
-rw-r--r--  1 william william  10994 Aug  8 01:57 num.rs
drwxrwxr-x 19 william william   4096 Aug  8 01:57 os
-rw-r--r--  1 william william  20453 Aug  8 01:57 panicking.rs
-rw-r--r--  1 william william  17288 Aug  8 01:57 panic.rs
-rw-r--r--  1 william william 129961 Aug  8 01:57 path.rs
drwxrwxr-x  2 william william   4096 Aug  8 01:57 prelude
-rw-r--r--  1 william william  36549 Aug  8 01:57 primitive_docs.rs
-rw-r--r--  1 william william  66214 Aug  8 01:57 process.rs
-rw-r--r--  1 william william   2138 Aug  8 01:57 rt.rs
drwxrwxr-x  3 william william   4096 Aug  8 01:57 sync
drwxrwxr-x 11 william william   4096 Aug  8 01:57 sys
drwxrwxr-x  2 william william   4096 Aug  8 01:57 sys_common
drwxrwxr-x  2 william william   4096 Aug  8 01:57 tests
drwxrwxr-x  2 william william   4096 Aug  8 01:57 thread
-rw-r--r--  1 william william  31341 Aug  8 01:57 time.rs
william@xubuntu-dtrain:~/Projects/embedded/msp430/AT2XT$

Attempt At Manual Fix

At this point, because I'm desperate, I try to manually force the src directory the msp430-fix toolchain is using to have the layout that cargo expects:

ln -s /home/william/Projects/toolchains/rust/library/std /home/william/Projects/toolchains/rust-msp430-fix/build-rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/src/rust/src/libstd
ln -s /home/william/Projects/toolchains/rust/library/core /home/william/Projects/toolchains/rust-msp430-fix/build-rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/src/rust/src/libcore
ln -s /home/william/Projects/toolchains/rust/library/alloc /home/william/Projects/toolchains/rust-msp430-fix/build-rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/src/rust/src/liballoc
ln -s /home/william/Projects/toolchains/rust/library/test /home/william/Projects/toolchains/rust-msp430-fix/build-rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/src/rust/src/libtest
mklink() { ln -s /home/william/Projects/toolchains/rust/library/$1 /home/william/Projects/toolchains/rust-msp430-fix/build-rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/src/rust/src/$1; }
mklink alloc
mklink core
mklink panic_abort
mklink panic_unwind
mklink unwind
mklink profiler_builtins
mklink proc_macro
mklink proc_macro
mklink std
mklink term

ln -s /home/william/Projects/toolchains/rust/library/rustc-std-workspace-core /home/william/Projects/toolchains/rust-msp430-fix/build-rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/src/rust/src/tools/rustc-std-workspace-core
ln -s /home/william/Projects/toolchains/rust/library/rustc-std-workspace-alloc /home/william/Projects/toolchains/rust-msp430-fix/build-rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/src/rust/src/tools/rustc-std-workspace-alloc
ln -s /home/william/Projects/toolchains/rust/library/rustc-std-workspace-std /home/william/Projects/toolchains/rust-msp430-fix/build-rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/src/rust/src/tools/rustc-std-workspace-std
mklink_tools() { ln -s /home/william/Projects/toolchains/rust/library/$1 /home/william/Projects/toolchains/rust-msp430-fix/build-rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/src/rust/src/tools/$1; }
mklink_tools core
mklink_tools std
mklink_tools alloc
mklink_tools panic_abort
mklink_tools unwind

At which point the build bombs with this:

cargo build -Zbuild-std=core --release --target=msp430-none-elf
error: package collision in the lockfile: packages alloc v0.0.0 (/home/william/Projects/toolchains/rust-msp430-fix/build-rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/src/rust/src/alloc) and alloc v0.0.0 (/home/william/Projects/toolchains/rust-msp430-fix/build-rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/src/rust/src/tools/alloc) are different, but only one can be written to lockfile unambiguously
error: Recipe `timer` failed on line 10 with exit code 101

At This Point, I Decide To Open This Issue

I'm kind of stuck here. I need to be able to build my own libcore for testing, so I can test a patch against a target that does not get libcore/libstd built by default (not built because it's a cross target?). But cargo is going out of it's way to prevent me from doing so. In the past xargo had an environment variable to set the source root to an external location (XARGO_RUST_SRC). However, it seems xargo has completely stopped working recently, and the build-std feature does not have an equivalent environment variable.

So... what am I doing wrong? Is there any possible workaround?

ehuss commented 3 years ago

Which version of nightly are you using? The directory names were rearranged recently, and src/libstd is now library/std. Make sure the version of nightly cargo you are using is in sync with the version of rustc in the git directory. If you have the latest master, the current nightly is 2020-08-08. You can also build cargo manually from the rust source tree, though it won't be linked into the toolchain automatically (you can sym link it manually if you want).

cr1901 commented 3 years ago

If you have the latest master, the current nightly is 2020-08-08.

I tried updating yesterday, but rustup kept choosing older versions because some components were missing (which is good!):

william@xubuntu-dtrain:~/Projects/embedded/msp430/AT2XT$ rustup update nightly
info: syncing channel updates for 'nightly-x86_64-unknown-linux-gnu'
info: latest update on 2020-08-08, rust version 1.47.0-nightly (09f4c9f50 2020-08-07)
info: skipping nightly which is missing installed component 'rustfmt-preview'
info: syncing channel updates for 'nightly-2020-08-07-x86_64-unknown-linux-gnu'
info: latest update on 2020-08-07, rust version 1.47.0-nightly (71f8d0c8f 2020-08-06)
cargo 1.47.0-nightly (aa6872140 2020-07-23)

I guess the directory names were changed between that window?

I can live without rustfmt for now (any way to install an old version even if it's mixing toolchains?). So I did:

rustup toolchain install nightly --force
cargo -V
cargo 1.47.0-nightly (1653f3546 2020-08-04)

I got a successful compile:

cargo build -Zbuild-std=core --release --target=msp430-none-elf
   Compiling compiler_builtins v0.1.32
   Compiling core v0.0.0 (/home/william/Projects/toolchains/rust-msp430-fix/build-rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/src/rust/library/core)
   Compiling autocfg v0.1.2
   Compiling rand_core v0.4.2
   Compiling semver-parser v0.7.0
   Compiling proc-macro2 v0.4.30
   Compiling libc v0.2.55
   Compiling unicode-xid v0.1.0
   Compiling syn v0.15.44
   Compiling msp430-rt v0.2.4
   Compiling bitflags v1.2.1
   Compiling msp430g2211 v0.2.1 (https://github.com/cr1901/msp430g2211?rev=6f39636#6f396364)
   Compiling rand_core v0.3.1
   Compiling rand_jitter v0.1.4
   Compiling semver v0.9.0
   Compiling rand_chacha v0.1.1
   Compiling rand_pcg v0.1.2
   Compiling rand v0.6.5
   Compiling rand_isaac v0.1.1
   Compiling rand_hc v0.1.0
   Compiling rand_xorshift v0.1.1
   Compiling rustc_version v0.2.3
   Compiling bare-metal v0.2.5
   Compiling quote v0.6.13
   Compiling rustc-std-workspace-core v1.99.0 (/home/william/Projects/toolchains/rust-msp430-fix/build-rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/src/rust/library/rustc-std-workspace-core)
   Compiling r0 v1.0.0
   Compiling vcell v0.1.0
   Compiling bit_reverse v0.1.7
   Compiling msp430-atomic v0.1.1 (https://github.com/cr1901/msp430-atomic?branch=llvm-asm#7e8f531d)
   Compiling once_cell v1.4.0
   Compiling rlibc v1.0.0
   Compiling msp430 v0.2.2
   Compiling panic-msp430 v0.2.0
   Compiling msp430-rt-macros v0.2.4
   Compiling at2xt v4.0.0 (/home/william/Projects/embedded/msp430/AT2XT)
    Finished release [optimized] target(s) in 21.19s
msp430-elf-objdump -Cd target/msp430-none-elf/release/at2xt > target/msp430-none-elf/release/at2xt.lst
msp430-elf-readelf -a --wide target/msp430-none-elf/release/at2xt > target/msp430-none-elf/release/at2xt.sym
msp430-elf-size target/msp430-none-elf/release/at2xt
   text    data     bss     dec     hex filename

I'm guessing the directory mismatch is going to affect a very small number of people, so feel free to close as invalid :).

ehuss commented 3 years ago

Ok, sounds like this is resolved.