search

rust-lang / rust

Empowering everyone to build reliable and efficient software.
https://www.rust-lang.org
Other
99.06k stars 12.8k forks source link

Cross-compilation x86_64-unknown-freebsd -> aarch64-unknown-freebsd failing at the Vendoring step fo `ar`: "failed to sync", "failed to write `vendor/ar/.cargo-checksum.json`" #80451

Open nguiard opened 3 years ago

nguiard commented 3 years ago

So I'm actually not sure that this is cross-compilation related, it appears when vendoring ar at the x.py dist step on a FreeBSD12.1-RELEASE-p1 VM. Kinda looks like a maybe simple No such file or directory, however I'm very new to building Rust so I'm not sure. Probably the most relevant output would be:

(last lines of output from x.py dist):

[...]
   Vendoring anyhow v1.0.32 (/home/nicolas/.cargo/registry/src/github.com-1ecc6299db9ec823/anyhow-1.0.32) to vendor/anyhow-1.0.32
   Vendoring anyhow v1.0.34 (/home/nicolas/.cargo/registry/src/github.com-1ecc6299db9ec823/anyhow-1.0.34) to vendor/anyhow
   Vendoring anymap v0.12.1 (/home/nicolas/.cargo/registry/src/github.com-1ecc6299db9ec823/anymap-0.12.1) to vendor/anymap
   Vendoring ar v0.8.0 (https://github.com/bjorn3/rust-ar.git?branch=do_not_remove_cg_clif_ranlib#de9ab0e5) (/home/nicolas/.cargo/git/checkouts/rust-ar-9b35aff8ad678e06/de9ab0e) to vendor/ar
error: failed to sync

Caused by:
  failed to write `vendor/ar/.cargo-checksum.json`

Caused by:
  No such file or directory (os error 2)

command did not execute successfully: "/usr/home/nicolas/Projects/rust/buildrust/rust/build/x86_64-unknown-freebsd/stage0/bin/cargo" "vendor" "--sync" "/usr/home/nicolas/Projects/rust/buildrust/rust/./src/tools/rust-analyzer/Cargo.toml" "/usr/home/nicolas/Projects/rust/buildrust/rust/./compiler/rustc_codegen_cranelift/Cargo.toml"
expected success, got: exit code: 101

failed to run: /usr/home/nicolas/Projects/rust/buildrust/rust/build/bootstrap/debug/bootstrap dist --config aarch64.toml -j 16
Build completed unsuccessfully in 0:07:05

Maybe that's sufficient for you guys to get an idea of the problem?

Much more (maybe unnecessary) details:

I'm trying to cross-compile on a FreeBSD 12.1-RELEASE-p1 (x86_64) VM for aarch64-unknown-freebsd. More precisely, I am trying to, at the same time, get a (x86_64 -> aarch64) and an (aarch64 -> aarch64) rustc, both compiled on x86_64-unknown-freebsd.

My build environment appears to be mostly set up right at this point, as most of the build is working, but I'm having a hard time completing it. I am unsure if the best place to ask is here or on the FreeBSD mailing lists.

It seems the build actually produced the rustc and std lib. I tested running both the x86_64 and aarch64 produced rustc with rustc --version on their respective platforms, and they run.

My steps

After setting up the FreeBSD build environment for aarch64 approximately as indicated here (approximately because the doc is a bit dated), the build step works fine with this command:

CC_aarch64_unknown_freebsd=aarch64-unknown-freebsd-clang \
CXX_aarch64_unknown_freebsd=aarch64-unknown-freebsd-clang++ \
python3.7 ./x.py build --config aarch64.toml -j 16

My aarch64.toml mostly follows the document linked, but that document dates from a few years back, so I had to modify the config a bit to overcome the first build issues.

changelog-seen = 2

[build]

host = ["x86_64-unknown-freebsd", "aarch64-unknown-freebsd"]

target = ["aarch64-unknown-freebsd"]

submodules = false

[llvm]
ldflags = "-lz"

[rust]
channel = "nightly"

# I commented this out. It was used in the doc linked above but made the compilation fail quickly
# saying that "host" was not expected here.
# The author talks about two patches he provided that I've been unable to find them. However, as
# the article is a bit old I figured maybe I'd try just commenting the block out, and it got me way further
# into the compilation.
#[host.aarch64-unknown-freebsd]
#cc = "aarch64-unknown-freebsd-clang"
#cxx = "aarch64-unknown-freebsd-clang++"

[target.aarch64-unknown-freebsd]
cc = "aarch64-unknown-freebsd-clang"
cxx = "aarch64-unknown-freebsd-clang++"

The dist step:

CC_aarch64_unknown_freebsd=aarch64-unknown-freebsd-clang \
CXX_aarch64_unknown_freebsd=aarch64-unknown-freebsd-clang++ \
python3.7 ./x.py build --config aarch64.toml -j 16

Produces this output:

    Finished dev [unoptimized + debuginfo] target(s) in 0.20s
Generating unstable book md files (aarch64-unknown-freebsd)
Building stage0 tool unstable-book-gen (x86_64-unknown-freebsd)
    Finished release [optimized] target(s) in 0.19s
Building stage0 tool rustbook (x86_64-unknown-freebsd)
    Finished release [optimized] target(s) in 0.23s
Rustbook (aarch64-unknown-freebsd) - unstable-book
Generating unstable book md files (x86_64-unknown-freebsd)
Building stage0 std artifacts (x86_64-unknown-freebsd -> x86_64-unknown-freebsd)
    Finished release [optimized] target(s) in 0.19s
Copying stage0 std from stage0 (x86_64-unknown-freebsd -> x86_64-unknown-freebsd /
x86_64-unknown-freebsd)
Building stage0 compiler artifacts (x86_64-unknown-freebsd -> x86_64-unknown-freebsd)
    Finished release [optimized] target(s) in 0.30s
Copying stage0 rustc from stage0 (x86_64-unknown-freebsd -> x86_64-unknown-freebsd
/ x86_64-unknown-freebsd)
Assembling stage1 compiler (x86_64-unknown-freebsd)
Building stage1 std artifacts (x86_64-unknown-freebsd -> x86_64-unknown-freebsd)
    Finished release [optimized] target(s) in 0.20s
Copying stage1 std from stage1 (x86_64-unknown-freebsd -> x86_64-unknown-freebsd /
x86_64-unknown-freebsd)
Building stage1 compiler artifacts (x86_64-unknown-freebsd -> x86_64-unknown-freebsd)
    Finished release [optimized] target(s) in 0.31s
Copying stage1 rustc from stage1 (x86_64-unknown-freebsd -> x86_64-unknown-freebsd
/ x86_64-unknown-freebsd)
Assembling stage2 compiler (x86_64-unknown-freebsd)
Documenting standalone (aarch64-unknown-freebsd)
Building rustdoc for stage2 (x86_64-unknown-freebsd)
    Finished release [optimized] target(s) in 0.20s
Documenting book redirect pages (aarch64-unknown-freebsd)
Documenting stage2 std (aarch64-unknown-freebsd)
Building stage1 std artifacts (x86_64-unknown-freebsd -> aarch64-unknown-freebsd)
    Finished release [optimized] target(s) in 0.20s
Copying stage1 std from stage1 (x86_64-unknown-freebsd -> x86_64-unknown-freebsd /
aarch64-unknown-freebsd)
Uplifting stage1 std (x86_64-unknown-freebsd -> aarch64-unknown-freebsd)
Copying stage2 std from stage1 (x86_64-unknown-freebsd -> x86_64-unknown-freebsd /
aarch64-unknown-freebsd)
    Finished release [optimized] target(s) in 0.45s
    Finished release [optimized] target(s) in 0.38s
    Finished release [optimized] target(s) in 0.50s
    Finished release [optimized] target(s) in 0.36s
    Finished release [optimized] target(s) in 0.39s
Documenting stage2 compiler (x86_64-unknown-freebsd)
        skipping - compiler/librustdoc docs disabled
Documenting stage2 compiler (aarch64-unknown-freebsd)
        skipping - compiler/librustdoc docs disabled
Documenting stage2 rustdoc (x86_64-unknown-freebsd)
        skipping - compiler/librustdoc docs disabled
Documenting stage2 rustdoc (aarch64-unknown-freebsd)
        skipping - compiler/librustdoc docs disabled
Documenting error index (x86_64-unknown-freebsd)
Building stage1 tool error_index_generator (x86_64-unknown-freebsd)
    Finished release [optimized] target(s) in 0.22s
Documenting error index (aarch64-unknown-freebsd)
Generating lint docs (x86_64-unknown-freebsd)
Uplifting stage1 std (x86_64-unknown-freebsd -> x86_64-unknown-freebsd)
Copying stage2 std from stage1 (x86_64-unknown-freebsd -> x86_64-unknown-freebsd /
x86_64-unknown-freebsd)
Building stage0 tool lint-docs (x86_64-unknown-freebsd)
    Finished release [optimized] target(s) in 0.17s
Rustbook (x86_64-unknown-freebsd) - rustc
Generating lint docs (aarch64-unknown-freebsd)
Rustbook (aarch64-unknown-freebsd) - rustc
Building stage0 tool fabricate (x86_64-unknown-freebsd)
    Finished release [optimized] target(s) in 0.21s
Dist rust-docs-nightly-aarch64-unknown-freebsd
        finished in 32.044 seconds
Dist rustc-nightly-x86_64-unknown-freebsd
        finished in 39.535 seconds
Building stage1 compiler artifacts (x86_64-unknown-freebsd -> aarch64-unknown-freebsd)
    Finished release [optimized] target(s) in 0.38s
Copying stage1 rustc from stage1 (x86_64-unknown-freebsd -> x86_64-unknown-freebsd
/ aarch64-unknown-freebsd)
Assembling stage2 compiler (aarch64-unknown-freebsd)
Building rustdoc for stage2 (aarch64-unknown-freebsd)
    Finished release [optimized] target(s) in 0.21s
Dist rustc-nightly-aarch64-unknown-freebsd
        finished in 37.977 seconds
Dist rust-std-nightly-aarch64-unknown-freebsd
        finished in 19.648 seconds
Dist rustc-dev-nightly-x86_64-unknown-freebsd
        finished in 109.446 seconds
Dist rustc-dev-nightly-aarch64-unknown-freebsd
        finished in 99.895 seconds
Dist rust-src-nightly
        finished in 7.656 seconds
   Vendoring addr2line v0.13.0 (/home/nicolas/.cargo/registry/src/github.com-1ecc6299db9ec823/addr2line-0.13.0) to vendor/addr2line-0.13.0
   Vendoring addr2line v0.14.0 (/home/nicolas/.cargo/registry/src/github.com-1ecc6299db9ec823/addr2line-0.14.0) to vendor/addr2line
   Vendoring adler v0.2.3 (/home/nicolas/.cargo/registry/src/github.com-1ecc6299db9ec823/adler-0.2.3) to vendor/adler
   Vendoring aho-corasick v0.7.13 (/home/nicolas/.cargo/registry/src/github.com-1ecc6299db9ec823/aho-corasick-0.7.13) to vendor/aho-corasick
   Vendoring ammonia v3.1.0 (/home/nicolas/.cargo/registry/src/github.com-1ecc6299db9ec823/ammonia-3.1.0) to vendor/ammonia
   Vendoring annotate-snippets v0.6.1 (/home/nicolas/.cargo/registry/src/github.com-1ecc6299db9ec823/annotate-snippets-0.6.1) to vendor/annotate-snippets-0.6.1
   Vendoring annotate-snippets v0.8.0 (/home/nicolas/.cargo/registry/src/github.com-1ecc6299db9ec823/annotate-snippets-0.8.0) to vendor/annotate-snippets
   Vendoring ansi_term v0.11.0 (/home/nicolas/.cargo/registry/src/github.com-1ecc6299db9ec823/ansi_term-0.11.0) to vendor/ansi_term-0.11.0
   Vendoring ansi_term v0.12.1 (/home/nicolas/.cargo/registry/src/github.com-1ecc6299db9ec823/ansi_term-0.12.1) to vendor/ansi_term
   Vendoring anyhow v1.0.32 (/home/nicolas/.cargo/registry/src/github.com-1ecc6299db9ec823/anyhow-1.0.32) to vendor/anyhow-1.0.32
   Vendoring anyhow v1.0.34 (/home/nicolas/.cargo/registry/src/github.com-1ecc6299db9ec823/anyhow-1.0.34) to vendor/anyhow
   Vendoring anymap v0.12.1 (/home/nicolas/.cargo/registry/src/github.com-1ecc6299db9ec823/anymap-0.12.1) to vendor/anymap
   Vendoring ar v0.8.0 (https://github.com/bjorn3/rust-ar.git?branch=do_not_remove_cg_clif_ranlib#de9ab0e5) (/home/nicolas/.cargo/git/checkouts/rust-ar-9b35aff8ad678e06/de9ab0e) to vendor/ar
error: failed to sync

Caused by:
  failed to write `vendor/ar/.cargo-checksum.json`

Caused by:
  No such file or directory (os error 2)

command did not execute successfully: "/usr/home/nicolas/Projects/rust/buildrust/rust/build/x86_64-unknown-freebsd/stage0/bin/cargo" "vendor" "--sync" "/usr/home/nicolas/Projects/rust/buildrust/rust/./src/tools/rust-analyzer/Cargo.toml" "/usr/home/nicolas/Projects/rust/buildrust/rust/./compiler/rustc_codegen_cranelift/Cargo.toml"
expected success, got: exit code: 101

failed to run: /usr/home/nicolas/Projects/rust/buildrust/rust/build/bootstrap/debug/bootstrap dist --config aarch64.toml -j 16
Build completed unsuccessfully in 0:07:05

Thanks a lot for your time. Any idea?

adam-azarchs commented 3 years ago

I am also running into this. Adding -v does nothing, to help debug, but looking at strace output it appears that what's going on is that, unlike for crates from the registry, with git sources it's not actually copying any files over from the git checkout into the vendor directory before it tries to create the .cargo-checksum.json, which means that the target vendor directory doesn't exist yet.