Support LLD linker for Darwin

[siddharthab]

Users can use with --linkopt=-fuse-ld=ld64.lld flag.

Eventually, we should make this the default. But only after we hear from some users that it works for their projects. This PR will make it easy for them to test.

[DavidZbarsky-at]

I gave this a quick spin, it might need more work for darwin_arm? (With llvm 17.0.6)

ERROR: /Users/david.zbarsky/h/source/hyperbase/server_shared/lightweight_gc_stats/BUILD.bazel:7:16: Linking server_shared/lightweight_gc_stats/libindex.node.so failed: (Exit 1): cc_wrapper.sh failed: error executing CppLink command (from target //server_shared/lightweight_gc_stats:libindex.node.so) external/llvm_toolchain/bin/cc_wrapper.sh @bazel-out/darwin_arm64-fastbuild/bin/server_shared/lightweight_gc_stats/libindex.node.so-2.params

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
ld64.lld: error: must specify -platform_version
ld64.lld: error: missing or unsupported -arch arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

(env-18.16.0) david.zbarsky@JF6FQ9PXP9 hyperbase % cat bazel-out/darwin_arm64-fastbuild/bin/server_shared/lightweight_gc_stats/libindex.node.so-2.params
-shared
-o
bazel-out/darwin_arm64-fastbuild/bin/server_shared/lightweight_gc_stats/libindex.node.so
bazel-out/darwin_arm64-fastbuild/bin/server_shared/lightweight_gc_stats/_objs/libindex.node.so/lightweight_gc_stats.o
-Wl,-S
-mmacos-version-min=14.2
-no-canonical-prefixes
-fobjc-link-runtime
--target=aarch64-apple-macosx
-lm
-no-canonical-prefixes
-fuse-ld=ld64.lld
-headerpad_max_install_names
-fobjc-link-runtime
-Lexternal/sysroot_darwin_universal//usr/lib
-lc++
-lc++abi
-undefined
dynamic_lookup
--sysroot=external/sysroot_darwin_universal/

[siddharthab]

It might.

I did develop this on an M2 Mini, but I tested it only on the simple tests in this repo, without a custom sysroot. The params look like this for me:

% cat bazel-out/darwin_arm64-fastbuild/bin/stdlib_test-2.params
-o
bazel-out/darwin_arm64-fastbuild/bin/stdlib_test
bazel-out/darwin_arm64-fastbuild/bin/_objs/stdlib_test/stdlib_test.o
bazel-out/darwin_arm64-fastbuild/bin/libstdlib.a
-Wl,-S
-mmacosx-version-min=10.11
-no-canonical-prefixes
-fobjc-link-runtime
--target=aarch64-apple-macosx
-lm
-no-canonical-prefixes
-fuse-ld=ld64.lld
-headerpad_max_install_names
-fobjc-link-runtime
-L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib
-lc++
-lc++abi
-Lexternal/toolchains_llvm~~llvm~llvm_toolchain_llvm/lib
-Wl,-t
--sysroot=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk

[siddharthab]

I found my mistake. At least the repo tests should be passing now.

[siddharthab]

I will wait for at least some people to say this change is working for them. This may not make it into the next release.

[DavidZbarsky-at]

@siddharthab I suspect this might be a llvm/lld version difference. Looks like the tests run on 16.x

[siddharthab]

On my local machine, I checked with LLVM 17 as well.

[aran]

I will wait for at least some people to say this change is working for them. This may not make it into the next release.

Anecdata, working fine on Bazel 7.1 on an M2 mac with up-to-date rules_rust, rules_go, llvm.

[siddharthab]

I think that for @DavidZbarsky-at , it is either the -shared or the universal sysroot that is causing issues. Based on the error message, it is likely to be the sysroot.

I am thinking of landing this behind a feature flag so people can more easily try it within their projects. Most likely, it might just mean adding a few more flags to the invocation which can be iterated on as more people try.

[siddharthab]

Made it optional, and added to the PR description instructions on how to use for a project.