tensorflow / text

Making text a first-class citizen in TensorFlow.
https://www.tensorflow.org/beta/tutorials/tensorflow_text/intro
Apache License 2.0
1.21k stars 333 forks source link

Build of 2.15.0 fails on ARM64 with linker error #1248

Closed SteffenBauer closed 4 months ago

SteffenBauer commented 4 months ago

I want to install keras-nlp on a Nvidia Jetson orin platform, which is ARM64 based. This has tensorflow-text as dependency. For earlier versions of tensorflow-text, building from source worked flawlessly on this platform. I wanted to upgrade to 2.15.0 now.

I already compiled and installed tensorflow 2.15.0 successfully from source on the platform. Unfortunately, compiling tensorflow-text branch v2.15.0 consistently fails with a linker error:

Compile output:

Using installed tensorflow.
$TEST_TMPDIR defined: output root default is '/tmp/bazel_cache/' and max_idle_secs default is '15'.
Starting local Bazel server and connecting to it...
WARNING: ignoring LD_PRELOAD in environment.
Updating setup.nightly.py to version 2.15.0
Updating __init__.py to version 2.15.0
Querying commit SHA
Found tensorflow commit sha: 6887368d6d4
Updating WORKSPACE file to use TensorFlow commit 6887368d6d46223f460358323c4b76d61d1558a8
$TEST_TMPDIR defined: output root default is '/tmp/bazel_cache/' and max_idle_secs default is '15'.
INFO: Reading 'startup' options from /tmp/workspace/text/.bazelrc: --windows_enable_symlinks
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=173
INFO: Reading rc options for 'build' from /tmp/workspace/text/.bazelrc:
  Inherited 'common' options: --experimental_repo_remote_exec
INFO: Reading rc options for 'build' from /tmp/workspace/text/.bazelrc:
  'build' options: --define framework_shared_object=true --define tsl_protobuf_header_only=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --features=-force_no_whole_archive --enable_platform_specific_config --define=with_xla_support=true --config=short_logs --config=v2 --define=no_aws_support=true --define=no_hdfs_support=true --experimental_cc_shared_library --experimental_link_static_libraries_once=false --incompatible_enforce_config_setting_visibility --action_env TF_HEADER_DIR=/home/steffen/.local/lib/python3.10/site-packages/tensorflow/include --action_env TF_SHARED_LIBRARY_DIR=/home/steffen/.local/lib/python3.10/site-packages/tensorflow --action_env TF_SHARED_LIBRARY_NAME=libtensorflow_framework.so.2 --action_env TF_CXX11_ABI_FLAG=1
INFO: Found applicable config definition build:short_logs in file /tmp/workspace/text/.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:v2 in file /tmp/workspace/text/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1
INFO: Found applicable config definition build:linux in file /tmp/workspace/text/.bazelrc: --host_copt=-w --copt=-Wno-all --copt=-Wno-extra --copt=-Wno-deprecated --copt=-Wno-deprecated-declarations --copt=-Wno-ignored-attributes --copt=-Wno-array-bounds --copt=-Wunused-result --copt=-Werror=unused-result --copt=-Wswitch --copt=-Werror=switch --copt=-Wno-error=unused-but-set-variable --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --cxxopt=-std=c++17 --host_cxxopt=-std=c++17 --config=dynamic_kernels --experimental_guard_against_concurrent_changes
INFO: Found applicable config definition build:dynamic_kernels in file /tmp/workspace/text/.bazelrc: --define=dynamic_loaded_kernels=true --copt=-DAUTOLOAD_DYNAMIC_KERNELS
DEBUG: Rule 'org_tensorflow' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "05b01f088e0da02118d28655061dacf0b007ec2d1af70640ec23685db8e20e9a"
DEBUG: Repository org_tensorflow instantiated at:
  /tmp/workspace/text/WORKSPACE:71:13: in <toplevel>
Repository rule http_archive defined at:
  /tmp/bazel_cache/_bazel_steffen/cf43a7396b71fa8084cec7d6db462663/external/bazel_tools/tools/build_defs/repo/http.bzl:372:31: in <toplevel>
DEBUG: /tmp/bazel_cache/_bazel_steffen/cf43a7396b71fa8084cec7d6db462663/external/org_tensorflow/third_party/repo.bzl:132:14: 
Warning: skipping import of repository 'icu' because it already exists.
DEBUG: /tmp/bazel_cache/_bazel_steffen/cf43a7396b71fa8084cec7d6db462663/external/org_tensorflow/third_party/repo.bzl:132:14: 
Warning: skipping import of repository 'pybind11' because it already exists.
INFO: Analyzed target //oss_scripts/pip_package:build_pip_package (194 packages loaded, 4897 targets configured).
INFO: Found 1 target...
ERROR: /tmp/workspace/text/tensorflow_text/core/pybinds/BUILD:154:17: Linking tensorflow_text/core/pybinds/pywrap_model_converter.so failed: (Exit 1): gcc failed: error executing command (from target //tensorflow_text/core/pybinds:pywrap_model_converter.so) /usr/bin/gcc @bazel-out/aarch64-opt/bin/tensorflow_text/core/pybinds/pywrap_model_converter.so-2.params
ld.lld: error: version script assignment of 'global' to symbol 'initpywrap_model_converter' failed: symbol not defined
ld.lld: error: version script assignment of 'global' to symbol 'init_pywrap_model_converter' failed: symbol not defined
ld.lld: error: version script assignment of 'global' to symbol 'ConvertSentencepieceModel' failed: symbol not defined
ld.lld: error: version script assignment of 'global' to symbol 'ConvertSentencepieceModelForDecoder' failed: symbol not defined
ld.lld: error: version script assignment of 'global' to symbol 'GetVocabularySize' failed: symbol not defined
collect2: error: ld returned 1 exit status
Target //oss_scripts/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 218.934s, Critical Path: 66.27s
INFO: 1077 processes: 203 internal, 874 local.
FAILED: Build did NOT complete successfully

Environment:

SteffenBauer commented 4 months ago

Update: Downgrading clang to 16.0.6 solved the issue. Closing this issue, as solved.