google / fleetbench

Benchmarking suite for Google workloads
Apache License 2.0
111 stars 9 forks source link

Bazel invokes `external/local_config_cc/None` as the linker #10

Open peterwaller-arm opened 1 year ago

peterwaller-arm commented 1 year ago

... if binutils not installed.

Note: This is not necessarily a fleetbench issue, but I want to at least document this here as I know others have hit similar issues, and it was a bit nasty to undo because it required knowing that you had to delete ~/.cache/bazel in order to get it working again once in the bad state. Ideally though, this scenario would work.

Context: I'm putting together a docker container with only a clang toolchain in it for the purposes of testing that toolchain, and I want to make sure there is no chance another toolchain is being picked up. If I build with --config=opt --config=clang and Clang/LLD in the $PATH, I get the following confusing error:

ERROR: /root/.cache/bazel/_bazel_root/9c087a2e32e91c2e0a08651c5d06d7a5/external/com_google_absl/absl/random/internal/BUILD.bazel:267:11: Linking external/com_google_absl/absl/random/internal/libplatform.a failed: (Exit 1): None failed: error executing command (from target @com_google_absl//absl/random/internal:platform)
  (cd /root/.cache/bazel/_bazel_root/9c087a2e32e91c2e0a08651c5d06d7a5/sandbox/linux-sandbox/328/execroot/com_google_fleetbench && \
  exec env - \
    PATH=/home/fleetbench/../compiler/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
    PWD=/proc/self/cwd \
  external/local_config_cc/None @bazel-out/aarch64-fastbuild-clang/bin/external/com_google_absl/absl/random/internal/libplatform.a-2.params)

That is, bazel appears to be is invoking external/local_config_cc/None which is not a valid path.

Furthermore, the error persists even after installing binutils, and even after a bazel clean. Removing the bazel cache directory or changing the bazel version does however allow the build to succeed once binutils is installed.

I found also that what seems to be going wrong is that it is checking for the presence of /usr/bin/ld.gold and hardcoding -fuse-ld=/usr/bin/ld.gold, even though that linker is ultimately unused, though I did also need to pass --linkopt=-fuse-ld=lld.

Supplying --features=thin_lto also did not work but supplying --{c,cxx,link}opt=-flto=thin along with -fuse-ld=lld did work.

rjogrady commented 1 year ago

Thanks for the note! We should clarify in our README all the required components. I didn't think about binutils. A nicer error message from Bazel wouldn't go amiss, of course.

It's possible that a 'bazel shutdown' would be sufficient for it to pick up the changes after you install new software. I tend to do bazel clean --expunge which cleans the cache and restarts the bazel server, perhaps a bit safer than manually removing the cache.

Are there two separate issues here? --features=thin_lto should be working with clang and bazel 6. I believe it will prefer ld.lld if it's in the path. Could you describe more what the problem was there?