Unable to build //src:java_tools_prebuilt_zip on MacOS with only Xcode command line tools

srdo-humio commented 7 months ago

Description of the bug:

I'm unable to build bazel build //src:java_tools_prebuilt_zip on MacOS with only the Xcode command line tools installed. It's unclear to me whether this is a bug, me needing to adjust Bazel's flags, or whether rules_graalvm's native_image simply requires a full Xcode install and can't be used with only the command line tools.

Which category does this issue belong to?

No response

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Have a MacOS device. Do not have Xcode installed. Install the Xcode command line tools via xcode-select –-install.

Run bazel build //src:java_tools_prebuilt_zip. It should fail.

Which operating system are you running Bazel on?

MacOS 14.4.1

What is the output of `bazel info release`?

release 7.1.1

If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

No response

What's the output of `git remote get-url origin; git rev-parse HEAD` ?

No response

Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.

No response

Have you found anything relevant by searching the web?

https://github.com/bazelbuild/bazel/issues/20638 https://github.com/bazelbuild/bazel/issues/20513#issuecomment-1861775111

Any other information, logs, or outputs that you want to share?

Here's some context on my machine, and what I've tried:

$ xcode-select -p
/Library/Developer/CommandLineTools
$ xcrun --show-sdk-version
11.3

Running bazel build //src:java_tools_prebuilt_zip on current master gets me

//src/java_tools/buildjar/java/com/google/devtools/build/java/turbine:turbine_direct_graal failed: I/O exception during sandboxed execution: xcrun failed with code 1.
This most likely indicates that SDK version [10.11] for platform [MacOSX] is unsupported for the target version of xcode.
Process exited with status 1
stdout: stderr: xcrun: error: SDK "macosx10.11" cannot be located
xcrun: error: SDK "macosx10.11" cannot be located

I figured it might be a problem with the --macos_minimum_os, so I edited the .bazelrc to say build:macos --macos_minimum_os=11.3 --host_macos_minimum_os=11.3 --macos_sdk_version=11.3. This gets me

bazel-out/darwin_x86_64-fastbuild/bin/src/java_tools/buildjar/java/com/google/devtools/build/java/turbine/src_java_tools_buildjar_java_com_google_devtools_build_java_turbine_turbine_direct_graal_processor_script_args.sh: line 13: DEVELOPER_DIR: unbound variable

I have tried adding common --repo_env=BAZEL_NO_APPLE_CPP_TOOLCHAIN=1 to the .bazelrc, and this makes no difference, with or without the other changes to the sdk version flags.

I have tried editing the MODULE.bazel file to ensure the apple_support line appears before rules_cc, since one of the linked issues implied order was important.

fmeum commented 7 months ago

rules_graalvm uses apple_support to set up the correct build environment for Xcode on macOS. As far as I understand, this requires the toolchain registered by apple_support, which in turn requires full Xcode and does not work with just the command-line tools.

@keith Is there a way to use apple_support.run in a way that also works with Bazel's auto-configured Unix toolchain?

keith commented 7 months ago

i think it's possible for apple_support.run to support CLT only, but it would require the unix toolchain to add some env vars which it doesn't today. and generally so far we've been trying to reduce the unix toolchain's knowledge of apple specific weirdnesses. maybe we could make that action fallback to just a normal run if it wasn't using xcode? I don't recall what xcode specific bits rules_graalvm needed

fmeum commented 7 months ago

It needs to pass through the Apple variables (mostly DEVELOPER_DIR) in a weird way because that's how GraalVM expects them. Is there a way to detect "if it wasn't using xcode" in the rule implementation function?

keith commented 7 months ago

i think in that case the xcode config apis from bazel return none. so theoretically we could use that. i think we'd have to still provide those variables, but at that point i guess we could safely hardcode them to where we "know" the CLT should be

rbeasley-avgo commented 4 months ago

I'm running into the same, except that, technically, I do have a complete Xcode installation—it just doesn't live beneath /Applications. In our case, we're treating everything like a standard Unix toolchain. We generate wrappers for Clang utils and provide them to Bazel via CC and CXX env vars. Additional environment variables (e.g. OTOOL, LIPO) point directly to /alternate/location/xcode-major.minor/Xcode.app/Developer/....

I'm using the Bazel CLI flags such as macos_minimum_os described above. I, too, run into the DEVELOPER_DIR: unbound variable error described above.

IIUC:

Bazel's MODULE.bazel names rules_cc before apple_support, so Bazel's chosen CC toolchain will be rules_cc's.
However, rules_graalvm assumes, if running on macOS, that the toolchain is provided by apple_support, and so it uses apple_support.run. I assume this is why @fmeum asked about a way to detect whether or not we're using Xcode.

To make progress locally, I'm patching Bazel's MODULE.bazel and adding a single_version_override(module_name = "rules_graalvm", patches=[...]).

What's the right path forward?

Should rules_graalvm condition its use of apple_support.run?
- Can rules_graalvm do this itself, or does apple_support need to provide an API for this?
- Similarly but alternatively, should apple_support.run internally check if the toolchain is an Xcode toolchain and behave accordingly?
Should rules_java/Bazel support opting out of GraalVM?
More ideas here.

fmeum commented 4 months ago

Since it's already asking a lot of rulesets to use apple_support for Xcode support on macOS, I think that this logic should ideally live in apple_support.run.

bazelbuild / bazel