search

elixir-nx / xla

Pre-compiled XLA extension
Apache License 2.0
83 stars 21 forks source link

Build error #21

Closed joelberkeley closed 1 year ago

joelberkeley commented 1 year ago

I'm getting the following error when trying to compile the binaries. Have you seen this before?

CC=/usr/bin/gcc-8 XLA_BUILD=true ./.github/scripts/publish_release.sh 
rm -f /home/joel/.cache/xla_extension/tf-3f878cff5b698b82eea85db2b60d65a2e320850e/tensorflow/compiler/xla/extension && \
    ln -s "/home/joel/git/joelberkeley/xla/extension" /home/joel/.cache/xla_extension/tf-3f878cff5b698b82eea85db2b60d65a2e320850e/tensorflow/compiler/xla/extension && \
    cd /home/joel/.cache/xla_extension/tf-3f878cff5b698b82eea85db2b60d65a2e320850e && \
    bazel build --define "framework_shared_object=false" -c opt    //tensorflow/compiler/xla/extension:xla_extension && \
    mkdir -p /home/joel/.cache/xla/0.3.0/cache/build/ && \
    cp -f /home/joel/.cache/xla_extension/tf-3f878cff5b698b82eea85db2b60d65a2e320850e/bazel-bin/tensorflow/compiler/xla/extension/xla_extension.tar.gz /home/joel/.cache/xla/0.3.0/cache/build/xla_extension-x86_64-linux-cpu.tar.gz
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=0 --terminal_columns=80
INFO: Reading rc options for 'build' from /home/joel/.cache/xla_extension/tf-3f878cff5b698b82eea85db2b60d65a2e320850e/.bazelrc:
  Inherited 'common' options: --experimental_repo_remote_exec
INFO: Reading rc options for 'build' from /home/joel/.cache/xla_extension/tf-3f878cff5b698b82eea85db2b60d65a2e320850e/.bazelrc:
  'build' options: --define framework_shared_object=true --java_toolchain=@tf_toolchains//toolchains/java:tf_java_toolchain --host_java_toolchain=@tf_toolchains//toolchains/java:tf_java_toolchain --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --enable_platform_specific_config --define=with_xla_support=true --config=short_logs --config=v2 --define=no_aws_support=true --define=no_hdfs_support=true --experimental_cc_shared_library --deleted_packages=tensorflow/compiler/mlir/tfrt,tensorflow/compiler/mlir/tfrt/benchmarks,tensorflow/compiler/mlir/tfrt/jit/python_binding,tensorflow/compiler/mlir/tfrt/jit/transforms,tensorflow/compiler/mlir/tfrt/python_tests,tensorflow/compiler/mlir/tfrt/tests,tensorflow/compiler/mlir/tfrt/tests/analysis,tensorflow/compiler/mlir/tfrt/tests/jit,tensorflow/compiler/mlir/tfrt/tests/lhlo_to_tfrt,tensorflow/compiler/mlir/tfrt/tests/tf_to_corert,tensorflow/compiler/mlir/tfrt/tests/tf_to_tfrt_data,tensorflow/compiler/mlir/tfrt/tests/saved_model,tensorflow/compiler/mlir/tfrt/transforms/lhlo_gpu_to_tfrt_gpu,tensorflow/core/runtime_fallback,tensorflow/core/runtime_fallback/conversion,tensorflow/core/runtime_fallback/kernel,tensorflow/core/runtime_fallback/opdefs,tensorflow/core/runtime_fallback/runtime,tensorflow/core/runtime_fallback/util,tensorflow/core/tfrt/common,tensorflow/core/tfrt/eager,tensorflow/core/tfrt/eager/backends/cpu,tensorflow/core/tfrt/eager/backends/gpu,tensorflow/core/tfrt/eager/core_runtime,tensorflow/core/tfrt/eager/cpp_tests/core_runtime,tensorflow/core/tfrt/fallback,tensorflow/core/tfrt/gpu,tensorflow/core/tfrt/run_handler_thread_pool,tensorflow/core/tfrt/runtime,tensorflow/core/tfrt/saved_model,tensorflow/core/tfrt/saved_model/tests,tensorflow/core/tfrt/tpu,tensorflow/core/tfrt/utils
INFO: Found applicable config definition build:short_logs in file /home/joel/.cache/xla_extension/tf-3f878cff5b698b82eea85db2b60d65a2e320850e/.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:v2 in file /home/joel/.cache/xla_extension/tf-3f878cff5b698b82eea85db2b60d65a2e320850e/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1
INFO: Found applicable config definition build:linux in file /home/joel/.cache/xla_extension/tf-3f878cff5b698b82eea85db2b60d65a2e320850e/.bazelrc: --copt=-w --host_copt=-w --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --cxxopt=-std=c++14 --host_cxxopt=-std=c++14 --config=dynamic_kernels --distinct_host_configuration=false --experimental_guard_against_concurrent_changes
INFO: Found applicable config definition build:dynamic_kernels in file /home/joel/.cache/xla_extension/tf-3f878cff5b698b82eea85db2b60d65a2e320850e/.bazelrc: --define=dynamic_loaded_kernels=true --copt=-DAUTOLOAD_DYNAMIC_KERNELS
Loading: 
Loading: 0 packages loaded
Analyzing: target //tensorflow/compiler/xla/extension:xla_extension (0 packages loaded, 0 targets configured)
Analyzing: target //tensorflow/compiler/xla/extension:xla_extension (6 packages loaded, 1 target configured)
Analyzing: target //tensorflow/compiler/xla/extension:xla_extension (6 packages loaded, 1 target configured)
Analyzing: target //tensorflow/compiler/xla/extension:xla_extension (6 packages loaded, 1 target configured)
Analyzing: target //tensorflow/compiler/xla/extension:xla_extension (6 packages loaded, 1 target configured)
Analyzing: target //tensorflow/compiler/xla/extension:xla_extension (6 packages loaded, 1 target configured)
Analyzing: target //tensorflow/compiler/xla/extension:xla_extension (6 packages loaded, 1 target configured)
Analyzing: target //tensorflow/compiler/xla/extension:xla_extension (6 packages loaded, 1 target configured)
Analyzing: target //tensorflow/compiler/xla/extension:xla_extension (6 packages loaded, 1 target configured)
Analyzing: target //tensorflow/compiler/xla/extension:xla_extension (6 packages loaded, 1 target configured)
DEBUG: Rule 'io_bazel_rules_docker' indicated that a canonical reproducible form can be obtained by modifying arguments shallow_since = "1596824487 -0400"
DEBUG: Repository io_bazel_rules_docker instantiated at:
  /home/joel/.cache/xla_extension/tf-3f878cff5b698b82eea85db2b60d65a2e320850e/WORKSPACE:23:14: in <toplevel>
  /home/joel/.cache/xla_extension/tf-3f878cff5b698b82eea85db2b60d65a2e320850e/tensorflow/workspace0.bzl:108:34: in workspace
  /home/joel/.cache/bazel/_bazel_joel/054737c50384c32ca43983f35a409061/external/bazel_toolchains/repositories/repositories.bzl:35:23: in repositories
Repository rule git_repository defined at:
  /home/joel/.cache/bazel/_bazel_joel/054737c50384c32ca43983f35a409061/external/bazel_tools/tools/build_defs/repo/git.bzl:199:33: in <toplevel>
Analyzing: target //tensorflow/compiler/xla/extension:xla_extension (7 packages loaded, 1 target configured)
Analyzing: target //tensorflow/compiler/xla/extension:xla_extension (7 packages loaded, 1 target configured)
Analyzing: target //tensorflow/compiler/xla/extension:xla_extension (7 packages loaded, 1 target configured)
Analyzing: target //tensorflow/compiler/xla/extension:xla_extension (75 packages loaded, 2244 targets configured)
Analyzing: target //tensorflow/compiler/xla/extension:xla_extension (175 packages loaded, 15501 targets configured)
Analyzing: target //tensorflow/compiler/xla/extension:xla_extension (179 packages loaded, 17607 targets configured)
INFO: Analyzed target //tensorflow/compiler/xla/extension:xla_extension (180 packages loaded, 18566 targets configured).
INFO: Found 1 target...
[0 / 27] [Prepa] Writing file tensorflow/compiler/xla/extension/xla_extension.args ... (5 actions, 0 running)
[7 / 32] checking cached actions
[156 / 1,323] checking cached actions
ERROR: /home/joel/.cache/bazel/_bazel_joel/054737c50384c32ca43983f35a409061/external/com_google_protobuf/BUILD:415:10: Compiling src/google/protobuf/compiler/main.cc failed: (Exit 1): gcc-8 failed: error executing command /usr/bin/gcc-8 -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections ... (remaining 32 argument(s) skipped)
gcc-8: error trying to exec 'cc1plus': execvp: No such file or directory
Target //tensorflow/compiler/xla/extension:xla_extension failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 36.232s, Critical Path: 0.62s
INFO: 376 processes: 364 internal, 12 local.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully
make: *** [Makefile:27: /home/joel/.cache/xla/0.3.0/cache/build/xla_extension-x86_64-linux-cpu.tar.gz] Error 1
** (Mix) Could not compile with "make" (exit status: 2).
You need to have gcc and make installed. If you are using
Ubuntu or any other Debian-based system, install the packages
"build-essential". Also install "erlang-dev" package if not
included in your Erlang/OTP version. If you're on Fedora, run
"dnf group install 'Development Tools'".

with

$ make -v
GNU Make 4.2.1
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2016 Free Software Foundation, Inc.
Licence GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
$ /usr/bin/gcc-8 -v
Using built-in specs.
COLLECT_GCC=/usr/bin/gcc-8
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/8/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 8.4.0-3ubuntu2' --with-bugurl=file:///usr/share/doc/gcc-8/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --prefix=/usr --with-gcc-major-version-only --program-suffix=-8 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 8.4.0 (Ubuntu 8.4.0-3ubuntu2) 
$ bazel --version
bazel 4.2.1
$ python3 -c "import numpy; print(numpy.__version__)"
1.23.1

I also did sudo apt install erlang-dev. Installed bazel with a .deb, not asdf

seanmor5 commented 1 year ago

I believe you need to install g++ to get around this error

joelberkeley commented 1 year ago

problem was also needed to install g++

joelberkeley commented 1 year ago

aha - agreed :p

worth adding to gcc section of README?

seanmor5 commented 1 year ago

Yes, feel free :)