google / cargo-raze

Generate Bazel BUILD from Cargo dependencies!
Apache License 2.0
478 stars 104 forks source link

macos: `ld: library not found for -lstdc++` when compiling `no_deps` example #247

Open nikclayton-dfinity opened 3 years ago

nikclayton-dfinity commented 3 years ago

Building a fresh checkout of examples/remote/no_deps with Bazel 3.6.0 fails on MacOS with the error:

% uname -a
Darwin Niks-MBP 19.6.0 Darwin Kernel Version 19.6.0: Thu Jun 18 20:49:00 PDT 2020; root:xnu-6153.141.1~1/RELEASE_X86_64 x86_64

% bazel build :all
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: SHA256 (https://static.rust-lang.org/dist/rustfmt-1.4.18-x86_64-apple-darwin.tar.gz) = 8c5ec426f2b55cd5de1a9cd47e98c5eeb7f24b7e8d83ef60673f1718952f706c
INFO: SHA256 (https://static.rust-lang.org/dist/rust-std-1.44.0-wasm32-unknown-unknown.tar.gz) = 8e12796a0c2fb083953042218f832bdeb78da1bfaf67b9dfe3d719920084d755
INFO: Analyzed target //remote/no_deps:no_deps (25 packages loaded, 171 targets configured).
INFO: Found 1 target...
INFO: From Compiling Rust bin no_deps (1 files):
error: linking with `external/local_config_cc/cc_wrapper.sh` failed: exit code: 1
  |
  = note: "external/local_config_cc/cc_wrapper.sh" "-m64" "-L" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps.no_deps.7rcbfp3g-cgu.0.rcgu.o" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps.no_deps.7rcbfp3g-cgu.1.rcgu.o" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps.no_deps.7rcbfp3g-cgu.2.rcgu.o" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps.no_deps.7rcbfp3g-cgu.3.rcgu.o" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps.no_deps.7rcbfp3g-cgu.4.rcgu.o" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps.no_deps.7rcbfp3g-cgu.5.rcgu.o" "-o" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps.20nzc483t9mjzk0n.rcgu.o" "-Wl,-dead_strip" "-nodefaultlibs" "-L" "external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib" "-L" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libstd-5aac390c25a3be91.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libpanic_unwind-8d95e919acff6358.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libhashbrown-b74724130e02bca1.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/librustc_std_workspace_alloc-0e0d29142d1c9272.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libbacktrace-f1652c422fbf7fc7.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libbacktrace_sys-3d74ad27c9a94aec.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/librustc_demangle-1255ec28ac7c09c4.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libunwind-8753dbda32b8facf.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libcfg_if-a00fd94477b6f959.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/liblibc-764e6791ca18d144.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/liballoc-a494dce1f057654b.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/librustc_std_workspace_core-2c78f494769c69d7.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libcore-47f37590dc106077.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libcompiler_builtins-edfd79f9969978bb.rlib" "-lSystem" "-lresolv" "-lc" "-lm" "-undefined" "dynamic_lookup" "-headerpad_max_install_names" "-lstdc++" "-lm"
  = note: ld: library not found for -lstdc++
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

Reproduction recipe:

  1. Install Bazel from Homebrew, following the instructions at https://docs.bazel.build/versions/3.6.0/install-os-x.html
% brew install bazel
...
==> bazel
Bash completion has been installed to:
  /usr/local/etc/bash_completion.d

zsh completions have been installed to:
  /usr/local/share/zsh/site-functions

% bazel --version
bazel 3.6.0-homebrew
  1. Install cargo-raze
% cargo --version
cargo 1.47.0 (f3c7e066a 2020-08-28)

% cargo install cargo-raze
...
   Installed package `cargo-raze v0.5.0` (executable `cargo-raze`)
  1. Check out the cargo-raze repository.
% git clone https://github.com/google/cargo-raze.git
Cloning into 'cargo-raze'...
remote: Enumerating objects: 6, done.
remote: Counting objects: 100% (6/6), done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 10825 (delta 0), reused 0 (delta 0), pack-reused 10819
Receiving objects: 100% (10825/10825), 10.23 MiB | 1.60 MiB/s, done.
Resolving deltas: 100% (7543/7543), done.

% git rev-parse HEAD
253a853d01d493bfaa35176510190aa09ec00128
  1. Attempt to build.
% cd cargo-raze/examples/remote/no_deps

% cargo raze
info: syncing channel updates for 'stable-x86_64-apple-darwin'
info: latest update on 2020-10-08, rust version 1.47.0 (18bf6b4f0 2020-10-07)
info: downloading component 'cargo'
info: downloading component 'clippy'
info: downloading component 'rust-docs'
info: downloading component 'rust-std'
info: downloading component 'rustc'
info: downloading component 'rustfmt'
info: installing component 'cargo'
info: Defaulting to 500.0 MiB unpack ram
info: installing component 'clippy'
info: installing component 'rust-docs'
info: installing component 'rust-std'
info: installing component 'rustc'
info: installing component 'rustfmt'
Loaded override settings: RazeSettings {
    workspace_path: "//remote/no_deps/cargo",
    incompatible_relative_workspace_path: true,
    target: "x86_64-unknown-linux-gnu",
    crates: {},
    gen_workspace_prefix: "remote_no_deps",
    genmode: Remote,
    output_buildfile_suffix: "BUILD.bazel",
    default_gen_buildrs: false,
}
Generated /Users/nik/github/cargo-raze/examples/remote/no_deps/cargo/remote/BUILD.bazel successfully
Generated /Users/nik/github/cargo-raze/examples/remote/no_deps/cargo/BUILD.bazel successfully
Generated /Users/nik/github/cargo-raze/examples/remote/no_deps/cargo/crates.bzl successfully

% bazel build :all
INFO: Analyzed target //remote/no_deps:no_deps (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
INFO: From Compiling Rust bin no_deps (1 files):
error: linking with `external/local_config_cc/cc_wrapper.sh` failed: exit code: 1
  |
  = note: "external/local_config_cc/cc_wrapper.sh" "-m64" "-L" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps.no_deps.7rcbfp3g-cgu.0.rcgu.o" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps.no_deps.7rcbfp3g-cgu.1.rcgu.o" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps.no_deps.7rcbfp3g-cgu.2.rcgu.o" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps.no_deps.7rcbfp3g-cgu.3.rcgu.o" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps.no_deps.7rcbfp3g-cgu.4.rcgu.o" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps.no_deps.7rcbfp3g-cgu.5.rcgu.o" "-o" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps.20nzc483t9mjzk0n.rcgu.o" "-Wl,-dead_strip" "-nodefaultlibs" "-L" "external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib" "-L" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libstd-5aac390c25a3be91.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libpanic_unwind-8d95e919acff6358.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libhashbrown-b74724130e02bca1.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/librustc_std_workspace_alloc-0e0d29142d1c9272.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libbacktrace-f1652c422fbf7fc7.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libbacktrace_sys-3d74ad27c9a94aec.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/librustc_demangle-1255ec28ac7c09c4.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libunwind-8753dbda32b8facf.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libcfg_if-a00fd94477b6f959.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/liblibc-764e6791ca18d144.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/liballoc-a494dce1f057654b.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/librustc_std_workspace_core-2c78f494769c69d7.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libcore-47f37590dc106077.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libcompiler_builtins-edfd79f9969978bb.rlib" "-lSystem" "-lresolv" "-lc" "-lm" "-undefined" "dynamic_lookup" "-headerpad_max_install_names" "-lstdc++" "-lm"
  = note: ld: library not found for -lstdc++
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

error: aborting due to previous error

ERROR: /Users/nik/github/cargo-raze/examples/remote/no_deps/BUILD:5:12: output 'remote/no_deps/no_deps' was not created
ERROR: /Users/nik/github/cargo-raze/examples/remote/no_deps/BUILD:5:12: not all outputs were created or valid
Target //remote/no_deps:no_deps failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.596s, Critical Path: 0.48s
INFO: 2 processes: 1 internal, 1 darwin-sandbox.
FAILED: Build did NOT complete successfully
nikclayton-dfinity commented 3 years ago

https://github.com/pandas-dev/pandas/issues/23424#issuecomment-446393981 may be relevant.

nikclayton-dfinity commented 3 years ago

https://developer.apple.com/documentation/xcode-release-notes/xcode-10-release-notes?preferredLanguage=occ suggests that on recent OSX libstd++ is no longer supported and libc++ should be used instead.

nikclayton-dfinity commented 3 years ago

https://github.com/bazelbuild/rules_rust/issues/226 is probably the real bug here.

nikclayton-dfinity commented 3 years ago

Add this to the bottom of the WORKSPACE file:

# Everything below here is key to building on OSX. Load a custom Clang
# toolchain rather than whatever happens to be installed on the machine.
#
# The entry in .bazelrc automatically selects this toolchain for building.
http_archive(
    name = "com_grail_bazel_toolchain",
    url = "https://github.com/grailbio/bazel-toolchain/archive/0.5.6.zip",
    sha256 = "759e6c10e7883b5969db01b5bdb01c5518bcf7ae637972d5a109eb994a669487",
    strip_prefix = "bazel-toolchain-0.5.6",
)

load("@com_grail_bazel_toolchain//toolchain:deps.bzl", "bazel_toolchain_dependencies")
bazel_toolchain_dependencies()

load("@com_grail_bazel_toolchain//toolchain:rules.bzl", "llvm_toolchain")
llvm_toolchain(
    name = "llvm_toolchain",
    absolute_paths = True,
    llvm_version = "9.0.0",
)

load("@llvm_toolchain//:toolchains.bzl", "llvm_register_toolchains")
llvm_register_toolchains()

and create .bazelrc in the same directory as the WORKSPACE file with the following contents:

build --crosstool_top=@llvm_toolchain//:toolchain

Then everything works for the no_deps example.

nikclayton-dfinity commented 3 years ago

complicated_cargo_library also works.

nikclayton-dfinity commented 3 years ago

non_cratesio doesn't.

It builds, but running it produces no output.

% cd examples/remote/non_cratesio

% cargo raze

% bazel build :non_cratesio_remote
INFO: Analyzed target //remote/non_cratesio:non_cratesio_remote (44 packages loaded, 563 targets configured).
INFO: Found 1 target...
Target //remote/non_cratesio:non_cratesio_remote up-to-date:
  bazel-bin/remote/non_cratesio/non_cratesio_remote
INFO: Elapsed time: 8.716s, Critical Path: 8.37s
INFO: 28 processes: 5 internal, 23 darwin-sandbox.
INFO: Build completed successfully, 28 total actions

% RUST_LOG=info bazel run :non_cratesio_remote
INFO: Analyzed target //remote/non_cratesio:non_cratesio_remote (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //remote/non_cratesio:non_cratesio_remote up-to-date:
  bazel-bin/remote/non_cratesio/non_cratesio_remote
INFO: Elapsed time: 0.091s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action

However, running it with Cargo has the same problem:

% RUST_LOG=info cargo run
warning: unused manifest key: raze
   Compiling compile_with_bazel v0.1.0 (/Users/nik/github/cargo-raze/examples/remote/non_cratesio)
    Finished dev [unoptimized + debuginfo] target(s) in 0.73s
     Running `target/debug/non_cratesio`

so I'm not sure what's going on there.

UebelAndre commented 3 years ago

Without digging too deeply, is this a bug in cargo-raze or in rules_rust? (as per https://github.com/google/cargo-raze/issues/247#issuecomment-707796112)

nikclayton-dfinity commented 3 years ago

The primary bug is in rules_rust.

I think there's probably two bugs in the cargo-raze examples.

  1. They don't work out of the box on macOS, you need to do https://github.com/bazelbuild/rules_rust/issues/226#issuecomment-626378800 in the WORKSPACE file.

The most version I have of those additions is:

http_archive(
    name = "com_grail_bazel_toolchain",
    url = "https://github.com/grailbio/bazel-toolchain/archive/0.5.6.zip",
    sha256 = "759e6c10e7883b5969db01b5bdb01c5518bcf7ae637972d5a109eb994a669487",
    strip_prefix = "bazel-toolchain-0.5.6",
)

load("@com_grail_bazel_toolchain//toolchain:deps.bzl", "bazel_toolchain_dependencies")
bazel_toolchain_dependencies()

load("@com_grail_bazel_toolchain//toolchain:rules.bzl", "llvm_toolchain")
llvm_toolchain(
    name = "llvm_toolchain",
    absolute_paths = True,
    llvm_version = "10.0.0",
)

load("@llvm_toolchain//:toolchains.bzl", "llvm_register_toolchains")
llvm_register_toolchains()
  1. non_cratesio producing no output irrespective of how it's built (https://github.com/google/cargo-raze/issues/247#issuecomment-708230957).
UebelAndre commented 3 years ago
1. They don't work out of the box on macOS, you need to do [bazelbuild/rules_rust#226 (comment)](https://github.com/bazelbuild/rules_rust/issues/226#issuecomment-626378800) in the `WORKSPACE` file.

Odd, I'm able to build the examples on MacOS just fine. Perhaps the rust rules are relying on some specific machine setup. It might be good to open a pull request to rules_rust making it's tooling more explicit so users see consistent behavior regardless of how their machines are configured (if that is indeed the case here).

2. `non_cratesio` producing no output irrespective of how it's built ([#247 (comment)](https://github.com/google/cargo-raze/issues/247#issuecomment-708230957)).

Interesting, seems to me like env_logger wasn't setup correctly? It does seem to be though. From some quick tests it doesn't seem like the logger is initializing to log Info level logs. Adding the following always panics

if !log_enabled!(log::Level::Info) {
  panic!("uh-oh");
}

This should definitely be looked at and updated. I may take a stab at it when I can find the time.

Sorry this isn't much help. But it's good to know where the issue is. You should chime in on https://github.com/bazelbuild/rules_rust/issues/226 and let the owners know users are still running into that issue 😄

keith commented 2 years ago

For folks experiencing ld: library not found for -lstdc++, this is a bug in bazel I've submitted a patch for, for more details see https://github.com/bazelbuild/bazel/issues/14395#issuecomment-1009490593

cdmistman commented 2 years ago

with the landing of bazelbuild/bazel#14750 I believe this issue can be closed