search

bazelbuild / bazel

a fast, scalable, multi-language and extensible build system
https://bazel.build
Apache License 2.0
23.13k stars 4.05k forks source link

the "cc-compiler-darwin_arm64" toolchain cannot be detected when install CommandLineTools and Xcode.app is not installed. #13514

Closed kekxv closed 3 years ago

kekxv commented 3 years ago

the "cc-compiler-darwin_arm64" toolchain cannot be detected when install CommandLineTools and Xcode.app is not installed.

What operating system are you running Bazel on?

MacOS(M1)

What's the output of bazel info release?

release 4.1.0

image

use bazel (/opt/homebrew/Cellar/bazel/4.1.0... (9 files, 29.9MB))

  bazel git:(master) bazel shutdown
Extracting Bazel installation...
  bazel git:(master) bazel build //examples/cpp/...
Starting local Bazel server and connecting to it...
DEBUG: /private/var/tmp/_bazel_caesar/3dbd51b498342f1222f4220c3d8fbd0a/external/bazel_toolchains/rules/rbe_repo/version_check.bzl:68:14:
Current running Bazel is ahead of bazel-toolchains repo. Please update your pin to bazel-toolchains repo in your WORKSPACE file.
DEBUG: /private/var/tmp/_bazel_caesar/3dbd51b498342f1222f4220c3d8fbd0a/external/bazel_toolchains/rules/rbe_repo/checked_in.bzl:103:14: rbe_ubuntu1804_java11 not using checked in configs as detect_java_home was set to True
DEBUG: /private/var/tmp/_bazel_caesar/3dbd51b498342f1222f4220c3d8fbd0a/external/bazel_toolchains/rules/rbe_repo/version_check.bzl:68:14:
Current running Bazel is ahead of bazel-toolchains repo. Please update your pin to bazel-toolchains repo in your WORKSPACE file.
DEBUG: /private/var/tmp/_bazel_caesar/3dbd51b498342f1222f4220c3d8fbd0a/external/bazel_toolchains/rules/rbe_repo/checked_in.bzl:103:14: rbe_ubuntu1604_java8 not using checked in configs as detect_java_home was set to True
ERROR: /Volumes/work/bazelbuild/bazel/examples/cpp/BUILD:9:10: Target '//examples/cpp:hello-world' depends on toolchain '@local_config_cc//:cc-compiler-darwin_arm64', which cannot be found: no such target '@local_config_cc//:cc-compiler-darwin_arm64': target 'cc-compiler-darwin_arm64' not declared in package '' defined by /private/var/tmp/_bazel_caesar/3dbd51b498342f1222f4220c3d8fbd0a/external/local_config_cc/BUILD'
ERROR: Analysis of target '//examples/cpp:hello-world' failed; build aborted: Analysis failed
INFO: Elapsed time: 44.611s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (18 packages loaded, 50 targets configured)
  bazel git:(master) git sho
  bazel git:(master) git pull
hint: Pulling without specifying how to reconcile divergent branches is
hint: discouraged. You can squelch this message by running one of the following
hint: commands sometime before your next pull:
hint:
hint:   git config pull.rebase false  # merge (the default strategy)
hint:   git config pull.rebase true   # rebase
hint:   git config pull.ff only       # fast-forward only
hint:
hint: You can replace "git config" with "git config --global" to set a default
hint: preference for all repositories. You can also pass --rebase, --no-rebase,
hint: or --ff-only on the command line to override the configured default per
hint: invocation.
remote: Enumerating objects: 103, done.
remote: Counting objects: 100% (103/103), done.
remote: Compressing objects: 100% (28/28), done.
remote: Total 72 (delta 49), reused 58 (delta 36), pack-reused 0
Unpacking objects: 100% (72/72), 9.63 KiB | 159.00 KiB/s, done.
From https://github.com/bazelbuild/bazel
   5a647bb..024deb5  master     -> origin/master
Updating 5a647bb..024deb5
Fast-forward
 src/main/java/com/google/devtools/build/lib/analysis/BaseRuleClasses.java                  | 12 ++++++---
 src/main/java/com/google/devtools/build/lib/analysis/RuleConfiguredTargetBuilder.java      |  2 +-
 src/main/java/com/google/devtools/build/lib/analysis/RuleContext.java                      |  4 +--
 .../com/google/devtools/build/lib/analysis/constraints/RuleContextConstraintSemantics.java |  2 +-
 src/main/java/com/google/devtools/build/lib/analysis/starlark/FunctionTransitionUtil.java  |  2 +-
 .../java/com/google/devtools/build/lib/analysis/starlark/StarlarkRuleClassFunctions.java   | 10 ++++++--
 src/main/java/com/google/devtools/build/lib/analysis/test/InstrumentedFilesCollector.java  |  2 +-
 src/main/java/com/google/devtools/build/lib/packages/Attribute.java                        | 29 +++++++++++++++++++++
 src/main/java/com/google/devtools/build/lib/packages/DependencyFilter.java                 |  7 +-----
 src/main/java/com/google/devtools/build/lib/packages/RuleClass.java                        | 20 +++++++++++----
 .../java/com/google/devtools/build/lib/analysis/StarlarkAttrTransitionProviderTest.java    | 61 ---------------------------------------------
 src/test/shell/integration/target_compatible_with_test.sh                                  | 16 +++++++++++-
 12 files changed, 83 insertions(+), 84 deletions(-)
  bazel git:(master) bazel build //examples/cpp/...
DEBUG: /private/var/tmp/_bazel_caesar/3dbd51b498342f1222f4220c3d8fbd0a/external/bazel_toolchains/rules/rbe_repo/version_check.bzl:68:14:
Current running Bazel is ahead of bazel-toolchains repo. Please update your pin to bazel-toolchains repo in your WORKSPACE file.
DEBUG: /private/var/tmp/_bazel_caesar/3dbd51b498342f1222f4220c3d8fbd0a/external/bazel_toolchains/rules/rbe_repo/checked_in.bzl:103:14: rbe_ubuntu1804_java11 not using checked in configs as detect_java_home was set to True
DEBUG: /private/var/tmp/_bazel_caesar/3dbd51b498342f1222f4220c3d8fbd0a/external/bazel_toolchains/rules/rbe_repo/version_check.bzl:68:14:
Current running Bazel is ahead of bazel-toolchains repo. Please update your pin to bazel-toolchains repo in your WORKSPACE file.
DEBUG: /private/var/tmp/_bazel_caesar/3dbd51b498342f1222f4220c3d8fbd0a/external/bazel_toolchains/rules/rbe_repo/checked_in.bzl:103:14: rbe_ubuntu1604_java8 not using checked in configs as detect_java_home was set to True
ERROR: /Volumes/work/bazelbuild/bazel/examples/cpp/BUILD:9:10: Target '//examples/cpp:hello-world' depends on toolchain '@local_config_cc//:cc-compiler-darwin_arm64', which cannot be found: no such target '@local_config_cc//:cc-compiler-darwin_arm64': target 'cc-compiler-darwin_arm64' not declared in package '' defined by /private/var/tmp/_bazel_caesar/3dbd51b498342f1222f4220c3d8fbd0a/external/local_config_cc/BUILD'
ERROR: Analysis of target '//examples/cpp:hello-world' failed; build aborted: Analysis failed
INFO: Elapsed time: 0.143s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (1 packages loaded, 3 targets configured)
  bazel git:(master)
kekxv commented 3 years ago

https://github.com/bazelbuild/bazel/issues/13509#issuecomment-847904966

kekxv commented 3 years ago

if you want work.you can chang the file $(bazel info output_base)/external/local_config_cc/BUILD:

vim $(bazel info output_base)/external/local_config_cc/BUILD
 48 cc_toolchain_suite(
 49     name = "toolchain",
 50     toolchains = {
 51         "darwin|clang": ":cc-compiler-darwin",
 52         "darwin_arm64|clang": ":cc-compiler-darwin",                // add
 53         "darwin": ":cc-compiler-darwin",
 54         "darwin_arm64": ":cc-compiler-darwin",                      // add
 55         "armeabi-v7a|compiler": ":cc-compiler-armeabi-v7a",
 56         "armeabi-v7a": ":cc-compiler-armeabi-v7a",
 57     },
 58 )
 60 cc_toolchain(                                  // add
 61     name = "cc-compiler-darwin_arm64",         // add
 62     toolchain_identifier = "local",            // add
 63     toolchain_config = ":local",               // add
 64     all_files = ":compiler_deps",              // add
 65     ar_files = ":compiler_deps",               // add
 66     as_files = ":compiler_deps",               // add
 67     compiler_files = ":compiler_deps",         // add
 68     dwp_files = ":empty",                      // add
 69     linker_files = ":compiler_deps",           // add
 70     objcopy_files = ":empty",                  // add
 71     strip_files = ":empty",                    // add
 72     supports_param_files = 1,                  // add
 73     module_map = ":module.modulemap",          // add
 74 )                                              // add
 75 cc_toolchain(
 76     name = "cc-compiler-darwin",
 77     toolchain_identifier = "local",
 78     toolchain_config = ":local",
 79     all_files = ":compiler_deps",
 80     ar_files = ":compiler_deps",
 81     as_files = ":compiler_deps",
 82     compiler_files = ":compiler_deps",
 83     dwp_files = ":empty",
 84     linker_files = ":compiler_deps",
 85     objcopy_files = ":empty",
 86     strip_files = ":empty",
 87     supports_param_files = 1,
 88     module_map = ":module.modulemap",
 89 )
meteorcloudy commented 3 years ago

The cause of this bug is that the cpu value darwin_arm64 didn't propagate correctly to the BUILD file. Will send a fix soon.

khanhtc1202 commented 3 years ago

Hi @meteorcloudy, is there any update on this issue, will it be fixed in v4.2? Just saw the PR is closed without merge. 👀

meteorcloudy commented 3 years ago

@khanhtc1202 The fix is imported at https://github.com/bazelbuild/bazel/commit/b4b0c321910bc968736ef48e8140528ea7d323cd, we don't just merge PR on github but import it to Google's internal codebase then export it github.

meteorcloudy commented 3 years ago

We'll have to cherry-pick this is 4.2 though.

juancresc commented 2 years ago

this is still an issue

juancresc commented 2 years ago

After doing what @kekxv suggested, I got this:

(mlp) ➜  tensorflow git:(master) ✗ bazel build --jobs=10 --compilation_mode=opt --copt=-march=native tensorflow  --verbose_failures        
WARNING: Option 'java_toolchain' is deprecated
WARNING: Option 'host_java_toolchain' is deprecated
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=176
INFO: Reading rc options for 'build' from /Users/mysuser/Documents/lib/tensorflow/.bazelrc:
  Inherited 'common' options: --experimental_repo_remote_exec
INFO: Reading rc options for 'build' from /Users/mysuser/Documents/lib/tensorflow/.bazelrc:
  'build' options: --define framework_shared_object=true --java_toolchain=//tensorflow/tools/toolchains/java:tf_java_toolchain --host_java_toolchain=//tensorflow/tools/toolchains/java:tf_java_toolchain --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --enable_platform_specific_config --define=with_xla_support=true --config=short_logs --config=v2 --define=no_aws_support=true --define=no_hdfs_support=true --experimental_cc_shared_library
INFO: Reading rc options for 'build' from /Users/mysuser/Documents/lib/tensorflow/.tf_configure.bazelrc:
  'build' options: --action_env PYTHON_BIN_PATH=/Users/mysuser/mambaforge/envs/mlp/bin/python3 --action_env PYTHON_LIB_PATH=/Users/mysuser/mambaforge/envs/mlp/lib/python3.8/site-packages --python_path=/Users/mysuser/mambaforge/envs/mlp/bin/python3
INFO: Reading rc options for 'build' from /Users/mysuser/Documents/lib/tensorflow/.bazelrc:
  'build' options: --deleted_packages=tensorflow/compiler/mlir/tfrt,tensorflow/compiler/mlir/tfrt/benchmarks,tensorflow/compiler/mlir/tfrt/jit/python_binding,tensorflow/compiler/mlir/tfrt/jit/transforms,tensorflow/compiler/mlir/tfrt/python_tests,tensorflow/compiler/mlir/tfrt/tests,tensorflow/compiler/mlir/tfrt/tests/analysis,tensorflow/compiler/mlir/tfrt/tests/jit,tensorflow/compiler/mlir/tfrt/tests/lhlo_to_tfrt,tensorflow/compiler/mlir/tfrt/tests/tf_to_corert,tensorflow/compiler/mlir/tfrt/tests/tf_to_tfrt_data,tensorflow/compiler/mlir/tfrt/tests/saved_model,tensorflow/compiler/mlir/tfrt/transforms/lhlo_gpu_to_tfrt_gpu,tensorflow/core/runtime_fallback,tensorflow/core/runtime_fallback/conversion,tensorflow/core/runtime_fallback/kernel,tensorflow/core/runtime_fallback/opdefs,tensorflow/core/runtime_fallback/runtime,tensorflow/core/runtime_fallback/util,tensorflow/core/tfrt/common,tensorflow/core/tfrt/eager,tensorflow/core/tfrt/eager/backends/cpu,tensorflow/core/tfrt/eager/backends/gpu,tensorflow/core/tfrt/eager/core_runtime,tensorflow/core/tfrt/eager/cpp_tests/core_runtime,tensorflow/core/tfrt/gpu,tensorflow/core/tfrt/run_handler_thread_pool,tensorflow/core/tfrt/runtime,tensorflow/core/tfrt/saved_model,tensorflow/core/tfrt/graph_executor,tensorflow/core/tfrt/saved_model/tests,tensorflow/core/tfrt/tpu,tensorflow/core/tfrt/utils
INFO: Found applicable config definition build:short_logs in file /Users/mysuser/Documents/lib/tensorflow/.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:v2 in file /Users/mysuser/Documents/lib/tensorflow/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1
INFO: Found applicable config definition build:macos in file /Users/mysuser/Documents/lib/tensorflow/.bazelrc: --apple_platform_type=macos --copt=-DGRPC_BAZEL_BUILD --copt=-w --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --cxxopt=-std=c++14 --host_cxxopt=-std=c++14
DEBUG: Rule 'io_bazel_rules_docker' indicated that a canonical reproducible form can be obtained by modifying arguments shallow_since = "1596824487 -0400"
DEBUG: Repository io_bazel_rules_docker instantiated at:
  /Users/mysuser/Documents/lib/tensorflow/WORKSPACE:23:14: in <toplevel>
  /Users/mysuser/Documents/lib/tensorflow/tensorflow/workspace0.bzl:110:34: in workspace
  /private/var/tmp/_bazel_mysuser/7d4fd1924864d2d6fce922e4404d7479/external/bazel_toolchains/repositories/repositories.bzl:35:23: in repositories
Repository rule git_repository defined at:
  /private/var/tmp/_bazel_mysuser/7d4fd1924864d2d6fce922e4404d7479/external/bazel_tools/tools/build_defs/repo/git.bzl:199:33: in <toplevel>
INFO: Analyzed target //tensorflow:tensorflow (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
ERROR: /private/var/tmp/_bazel_mysuser/7d4fd1924864d2d6fce922e4404d7479/external/com_google_absl/absl/types/BUILD.bazel:153:11: Compiling absl/types/bad_optional_access.cc [for host] failed: missing input file 'external/local_config_cc/module.modulemap', owner: '@local_config_cc//:module.modulemap'
ERROR: /private/var/tmp/_bazel_mysuser/7d4fd1924864d2d6fce922e4404d7479/external/com_google_absl/absl/types/BUILD.bazel:153:11: Compiling absl/types/bad_optional_access.cc [for host] failed: 1 input file(s) do not exist
Target //tensorflow:tensorflow failed to build
ERROR: /Users/mysuser/Documents/lib/tensorflow/tensorflow/core/framework/BUILD:1400:31 Middleman _middlemen/_S_Stensorflow_Score_Sframework_Cattr_Uvalue_Uproto_Utext-BazelCppSemantics_build_arch_darwin_arm64-opt failed: 1 input file(s) do not exist
INFO: Elapsed time: 0.337s, Critical Path: 0.03s
INFO: 11 processes: 11 internal.
FAILED: Build did NOT complete successfully
keith commented 2 years ago

Your error here is very different:

ERROR: /private/var/tmp/_bazel_mysuser/7d4fd1924864d2d6fce922e4404d7479/external/com_google_absl/absl/types/BUILD.bazel:153:11: Compiling absl/types/bad_optional_access.cc [for host] failed: missing input file 'external/local_config_cc/module.modulemap', owner: '@local_config_cc//:module.modulemap'
ERROR: /private/var/tmp/_bazel_mysuser/7d4fd1924864d2d6fce922e4404d7479/external/com_google_absl/absl/types/BUILD.bazel:153:11: Compiling absl/types/bad_optional_access.cc [for host] failed: 1 input file(s) do not exist
Target //tensorflow:tensorflow failed to build
ERROR: /Users/mysuser/Documents/lib/tensorflow/tensorflow/core/framework/BUILD:1400:31 Middleman _middlemen/_S_Stensorflow_Score_Sframework_Cattr_Uvalue_Uproto_Utext-BazelCppSemantics_build_arch_darwin_arm64-opt failed: 1 input file(s) do not exist
juancresc commented 2 years ago

@keith indeed

keith commented 2 years ago

It looks like something else about your environment might have issues, can you verify the command line tools are up to date? You might also try cleaning the workspace after you update them. Otherwise can you file a new issue with repro steps?

SuperBo commented 2 years ago

@juancrescente , Try using bazel 5.0.0 instead of 4.2.2 when building tensorflow. I think the fix is merged into 5.0.0 version, not 4.2.2