broken
commented
3 years ago I think this error appears when using bazel 4.0.0 (https://github.com/bazelbuild/bazel/issues/12887). We currently build with Bazel 3.7.2. Does changing that help?
Closed dbl001 closed 2 years ago
broken
commented
3 years ago I think this error appears when using bazel 4.0.0 (https://github.com/bazelbuild/bazel/issues/12887). We currently build with Bazel 3.7.2. Does changing that help?
dbl001
commented
3 years ago With Bazel 3.7.2 and clang: % clang --version Apple clang version 12.0.5 (clang-1205.0.22.9) Target: x86_64-apple-darwin20.5.0 Thread model: posix InstalledDir: /Library/Developer/CommandLineTools/usr/bin
ERROR: /Users/davidlaxer/text/tensorflow_text/core/kernels/BUILD:355:23: C++ compilation of rule '//tensorflow_text/core/kernels:sentencepiece_kernels' failed (Exit 1): cc_wrapper.sh failed: error executing command external/local_config_cc/cc_wrapper.sh -U_FORTIFY_SOURCE -fstack-protector -Wall -Wthread-safety -Wself-assign -fcolor-diagnostics -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG ... (remaining 53 argument(s) skipped)
tensorflow_text/core/kernels/sentencepiece_kernels.cc:70:65: error: only virtual member functions can be marked 'override'
Status AsGraphDef(GraphDefBuilder* builder, Node** out) const override {In resource_mgr.h Tensorflow 2.4.1 the class ResourceBase : public core::RefCounted AsGraphDef is not defined as it in Tensorflow 2.5.0 :
virtual Status AsGraphDef(GraphDefBuilder* builder, Node** out) const { return errors::Unimplemented("AsGraphDef not implemented for resource ", DebugString()); }
The Apple Machine Learning version of tensorflow with GPU support for AMD Radeon is based upon tensorflow 2.4.1. Any ideas?
dbl001
commented
3 years ago I cloned the 2.4 tensorflow_text branch and built with bazel 3.7.2
In [1]: import tensorflow_text
---------------------------------------------------------------------------
NotFoundError Traceback (most recent call last)
<ipython-input-1-4dd33f1b1ed3> in <module>
----> 1 import tensorflow_text
~/text/tensorflow_text/__init__.py in <module>
19 # pylint: disable=wildcard-import
20 from tensorflow_text.python import keras
---> 21 from tensorflow_text.python import metrics
22 from tensorflow_text.python.ops import *
23
~/text/tensorflow_text/python/metrics/__init__.py in <module>
18
19 # pylint: disable=wildcard-import
---> 20 from tensorflow_text.python.metrics.text_similarity_metric_ops import *
21
22 # Public symbols in the "tensorflow_text.metrics" package.
~/text/tensorflow_text/python/metrics/text_similarity_metric_ops.py in <module>
26 from tensorflow.python.framework import load_library
27 from tensorflow.python.platform import resource_loader
---> 28 gen_text_similarity_metric_ops = load_library.load_op_library(resource_loader.get_path_to_datafile('_text_similarity_metric_ops.dylib'))
29
30
~/anaconda3/envs/tensorflow_mac/lib/python3.8/site-packages/tensorflow/python/framework/load_library.py in load_op_library(library_filename)
55 RuntimeError: when unable to load the library or get the python wrappers.
56 """
---> 57 lib_handle = py_tf.TF_LoadLibrary(library_filename)
58 try:
59 wrappers = _pywrap_python_op_gen.GetPythonWrappers(
NotFoundError: dlopen(/Users/davidlaxer/text/tensorflow_text/python/metrics/_text_similarity_metric_ops.dylib, 6): Symbol not found: __ZN10tensorflow15TensorShapeBaseINS_11TensorShapeEEC2EN4absl14lts_2020_09_234SpanIKxEE
Referenced from: /Users/davidlaxer/text/tensorflow_text/python/metrics/_text_similarity_metric_ops.dylib
Expected in: /Users/davidlaxer/anaconda3/envs/tensorflow_mac/lib/python3.8/site-packages/tensorflow/python/../libtensorflow_framework.2.dylib
in /Users/davidlaxer/text/tensorflow_text/python/metrics/_text_similarity_metric_ops.dylibI copied these shared library files from the Apple Tensorflow 2.4.1 environment libraries:
E.g. /Users/davidlaxer/anaconda3/envs/tensorflow_mac/lib/python3.8/site-packages/tensorflow_text/python/ops/*.dylib
ls -l /Users/davidlaxer/text/tensorflow_text/python/ops/*.dylib
-rwxr-xr-x 1 davidlaxer staff 137248 Jun 12 14:05 /Users/davidlaxer/text/tensorflow_text/python/ops/_constrained_sequence_op.dylib
-rwxr-xr-x 1 davidlaxer staff 134760 Jun 12 14:05 /Users/davidlaxer/text/tensorflow_text/python/ops/_mst_ops.dylib
-rwxr-xr-x 1 davidlaxer staff 2267592 Jun 12 14:05 /Users/davidlaxer/text/tensorflow_text/python/ops/_normalize_ops.dylib
-rwxr-xr-x 1 davidlaxer staff 563648 Jun 12 14:05 /Users/davidlaxer/text/tensorflow_text/python/ops/_regex_split_ops.dylib
-rwxr-xr-x 1 davidlaxer staff 969352 Jun 12 14:05 /Users/davidlaxer/text/tensorflow_text/python/ops/_sentence_breaking_ops.dylib
-rwxr-xr-x 1 davidlaxer staff 2273704 Jun 12 14:05 /Users/davidlaxer/text/tensorflow_text/python/ops/_sentencepiece_tokenizer.dylib
-rwxr-xr-x 1 davidlaxer staff 310264 Jun 12 14:05 /Users/davidlaxer/text/tensorflow_text/python/ops/_split_merge_from_logits_tokenizer.dylib
-rwxr-xr-x 1 davidlaxer staff 310840 Jun 12 14:05 /Users/davidlaxer/text/tensorflow_text/python/ops/_split_merge_tokenizer.dylib
-rwxr-xr-x 1 davidlaxer staff 722216 Jun 12 14:05 /Users/davidlaxer/text/tensorflow_text/python/ops/_state_based_sentence_breaker_op.dylib
-rwxr-xr-x 1 davidlaxer staff 255656 Jun 12 14:05 /Users/davidlaxer/text/tensorflow_text/python/ops/_unicode_script_tokenizer.dylib
-rwxr-xr-x 1 davidlaxer staff 235976 Jun 12 14:05 /Users/davidlaxer/text/tensorflow_text/python/ops/_whitespace_tokenizer.dylib
-rwxr-xr-x 1 davidlaxer staff 129008 Jun 12 14:05 /Users/davidlaxer/text/tensorflow_text/python/ops/_wordpiece_tokenizer.dylibTensorflow_text is running.
broken
commented
3 years ago Great! I'm happy to hear you got this running.
We will be looking at trying to add the new mac targets soon.
broken
commented
3 years ago Also, thanks for including your steps and solution for others.
awav
commented
2 years ago @broken,
Hi, I would like to use tf-text with latest tensorflow build (from source) on Linux. However, I'm getting similar error:
(py37) artem@blabla:~/code/text$ ./oss_scripts/run_build.sh
++ uname -s
+ osname=Linux
+ [[ Linux == \D\a\r\w\i\n ]]
+ ./oss_scripts/configure.sh
linux
WARNING: Package(s) not found: tensorflow
Using installed tensorflow.
+ bazel build --enable_runfiles oss_scripts/pip_package:build_pip_package
Starting local Bazel server and connecting to it...
DEBUG: /1TB/artem/.cache/bazel/_bazel_artem/513dad259ce460b27c1f370613b4b9a5/external/org_tensorflow/third_party/repo.bzl:109:14:
Warning: skipping import of repository 'icu' because it already exists.
DEBUG: /1TB/artem/.cache/bazel/_bazel_artem/513dad259ce460b27c1f370613b4b9a5/external/org_tensorflow/third_party/repo.bzl:109:14:
Warning: skipping import of repository 'pybind11' because it already exists.
DEBUG: /1TB/artem/.cache/bazel/_bazel_artem/513dad259ce460b27c1f370613b4b9a5/external/tf_runtime/third_party/cuda/dependencies.bzl:51:10: The following command will download NVIDIA proprietary software. By using the software you agree to comply with the terms of the license agreement that accompanies the software. If you do not agree to the terms of the license agreement, do not use the software.
WARNING: /1TB/artem/.cache/bazel/_bazel_artem/513dad259ce460b27c1f370613b4b9a5/external/local_config_tf/BUILD:10:11: in linkstatic attribute of cc_library rule @local_config_tf//:libtensorflow_framework: setting 'linkstatic=1' is recommended if there are no object files
ERROR: /1TB/artem/.cache/bazel/_bazel_artem/513dad259ce460b27c1f370613b4b9a5/external/com_google_protobuf/BUILD:981:21: in proto_lang_toolchain rule @com_google_protobuf//:cc_toolchain: '@com_google_protobuf//:cc_toolchain' does not have mandatory provider 'ProtoInfo'.
INFO: Repository com_google_sentencepiece instantiated at:
/home/artem/code/text/WORKSPACE:5:13: in <toplevel>
Repository rule http_archive defined at:
/1TB/artem/.cache/bazel/_bazel_artem/513dad259ce460b27c1f370613b4b9a5/external/bazel_tools/tools/build_defs/repo/http.bzl:336:31: in <toplevel>
ERROR: Analysis of target '//oss_scripts/pip_package:build_pip_package' failed; build aborted: Analysis of target '@com_google_protobuf//:cc_toolchain' failed
INFO: Elapsed time: 4.845s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (61 packages loaded, 2040 targets configured)Linux blabla 5.4.0-81-generic #91~18.04.1-Ubuntu SMP Fri Jul 23 13:36:29 UTC 2021 x86_64 x86_64 x86_64 GNU/LinuxFixed,
3.7.2bazel call to bazelisk in _runbuild.sh
jhkang-illunex
commented
2 years ago on lastest version
prepare_tf_dep.sh had som issue on m1 apple
sed -i "s/project_version = 'REPLACE_ME'/project_version = '${tf_version}'/" oss_scripts/pip_package/setup.nightly.py
short_commit_sha=$($installed_python -c 'import tensorflow as tf; print(tf.__git_version__)' | tail -1 | grep -oP '(?<=-g)[0-9a-f]*$')
and other line..
that shell run on linux but apple cannot run..
broken
commented
2 years ago You should be able to safely skip running it, and I'll update the build script to skip over running it in the m1 case.
The prepare_tf_dep script's main purpose is to build against the exact version of TF you have installed and set the package version for tf-text. However, tensorflow-macos does not set the __git_version__ used by the script, does not release nightlies, and builds against it aren't for pypi uploaded packages, so it shouldn't be needed for these cases.
I am trying to build tensorflow-text in a condo environment with an Apple Machine Learning version of Tensorflow on Big Sur on an Intel iMac w/AMD Radeon GPU.
I am using bazelisk E.g. $ bazelisk build --enable_runfiles oss_scripts/pip_package:build_pip_package