search

tensorflow / io

Dataset, streaming, and file system extensions maintained by TensorFlow SIG-IO
Apache License 2.0
704 stars 284 forks source link

Cannot build from source for macos arm64 M1/M2 #1750

Open nmatare opened 1 year ago

nmatare commented 1 year ago

Hello,

I'm trying to build from source using the guide here. Following the same pattern as the build scripts, I do the below, but fail with the following bazel error:

Workflow:

git clone -b v0.28.0 git@github.com:tensorflow/io.git

cd io

rm -f .bazelrc
python -m pip install tensorflow-macos==2.11.0
python -m pip install tensorflow-metal==0.7.0

$PYTHON tools/build/configure.py

bazel build -s --verbose_failures $BAZEL_OPTIMIZATION //tensorflow_io/... //tensorflow_io_gcs_filesystem/...
**Error output:** ```sh ERROR: /private/var/tmp/_bazel_johnconnor/31d41367353a93cdf74da2b122b69682/external/com_github_cares_cares/BUILD.bazel:109:11: Compiling ares_getnameinfo.c failed: (Aborted): sandbox-exec failed: error executing command (cd /private/var/tmp/_bazel_johnconnor/31d41367353a93cdf74da2b122b69682/sandbox/darwin-sandbox/746/execroot/org_tensorflow_io && \ exec env - \ APPLE_SDK_PLATFORM=MacOSX \ APPLE_SDK_VERSION_OVERRIDE=13.1 \ DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer \ PATH=/Users/johnconnor/Library/Caches/bazelisk/downloads/bazelbuild/bazel-5.1.1-darwin-arm64/bin:/Users/johnconnor/downloads/google-cloud-sdk/bin:/Users/johnconnor/opt/miniconda3/envs/build-test/bin:/Users/johnconnor/opt/miniconda3/condabin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin \ SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.1.sdk \ TF_HEADER_DIR=/Users/johnconnor/opt/miniconda3/envs/build-test/lib/python3.8/site-packages/tensorflow/include \ TF_SHARED_LIBRARY_DIR=/Users/johnconnor/opt/miniconda3/envs/build-test/lib/python3.8/site-packages/tensorflow \ TF_SHARED_LIBRARY_NAME=libtensorflow_framework.2.dylib \ TMPDIR=/var/folders/s7/8qr5g66d4gd68qxhhm48y0v40000gp/T/ \ XCODE_VERSION_OVERRIDE=14.2.0.14C18 \ ZERO_AR_DATE=1 \ /usr/bin/sandbox-exec -f /private/var/tmp/_bazel_johnconnor/31d41367353a93cdf74da2b122b69682/sandbox/darwin-sandbox/746/sandbox.sb /var/tmp/_bazel_johnconnor/install/e301fb5eb5984c19c92f75b75e58c08b/process-wrapper '--timeout=0' '--kill_delay=15' external/local_config_cc/wrapped_clang '-D_FORTIFY_SOURCE=1' -fstack-protector -fcolor-diagnostics -Wall -Wthread-safety -Wself-assign -fno-omit-frame-pointer -O0 -DDEBUG 'DEBUG_PREFIX_MAP_PWD=.' -iquote external/com_github_cares_cares -iquote bazel-out/darwin_arm64-fastbuild/bin/external/com_github_cares_cares -isystem external/com_github_cares_cares -isystem bazel-out/darwin_arm64-fastbuild/bin/external/com_github_cares_cares -MD -MF bazel-out/darwin_arm64-fastbuild/bin/external/com_github_cares_cares/_objs/ares/ares_getnameinfo.d -DCARES_STATICLIB '-frandom-seed=bazel-out/darwin_arm64-fastbuild/bin/external/com_github_cares_cares/_objs/ares/ares_getnameinfo.o' -isysroot __BAZEL_XCODE_SDKROOT__ -F__BAZEL_XCODE_SDKROOT__/System/Library/Frameworks -F__BAZEL_XCODE_DEVELOPER_DIR__/Platforms/MacOSX.platform/Developer/Library/Frameworks '-mmacosx-version-min=13.1' -no-canonical-prefixes -pthread '-fvisibility=hidden' '-D_GLIBCXX_USE_CXX11_ABI=0' '-DEIGEN_MAX_ALIGN_BYTES=64' -DGRPC_BAZEL_BUILD '-mmacosx-version-min=10.14' -Wunguarded-availability -D_GNU_SOURCE '-D_HAS_EXCEPTIONS=0' -DHAVE_CONFIG_H -no-canonical-prefixes -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -target arm64-apple-macosx -c external/com_github_cares_cares/ares_getnameinfo.c -o bazel-out/darwin_arm64-fastbuild/bin/external/com_github_cares_cares/_objs/ares/ares_getnameinfo.o) external/com_github_cares_cares/ares_getnameinfo.c:306:15: error: implicit declaration of function 'getservbyport_r' is invalid in C99 [-Werror,-Wimplicit-function-declaration] if (getservbyport_r(port, proto, &se, (void *)tmpbuf, ^ external/com_github_cares_cares/ares_getnameinfo.c:306:15: note: did you mean 'getservbyport'? /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.1.sdk/usr/include/netdb.h:287:17: note: 'getservbyport' declared here struct servent *getservbyport(int, const char *); ^ 1 error generated. Error in child process '/usr/bin/xcrun'. 1 ```

Xcode version

xcodebuild -version
Xcode 14.2
Build version 14C18

Bazel version

bazel --version
bazel 5.1.1

Python version

python --version
Python 3.8.15

Related

https://github.com/tensorflow/io/issues/1298

https://github.com/tensorflow/io/issues/1643

Thanks!

Algomorph commented 6 months ago

In case anyone stumbles upon this (I did after upgrade to XCode 15.3 on an M1):

this bug stems from darwin_arm64 cpu configuration missing from the cares.BUILD file in third_party/cares folder of the gRPC version that TensorFlow is still using at time of writing (Jan 2020 or so, I believe). gRPC has long since included that fix (here).

I patched commit 4bdc149ac84738b06a592473595c1c9c2bd2a9a3 of TensorFlow, because that's the one MediaPipe 0.10.11 is using (most recent release at time of writing).

Here's the patch: tensorflow_add_grpc_cares_darwin_arm64_support.patch

Until the gRPC used by TernsorFlow changes (and, if it does, I'm hoping it will include the corrected grpc/C-Ares build), you can easily recreate the patch for whatever version of TensorFlow you are using by checking out that commit into a separate branch, applying the patch, commiting, and then rebasing the branch on whatever commit or tag or branch of TensorFlow you need. (git show HEAD > patch_file_name.diff command can be used to make the patch file out of the previous commit, for anyone not familiar.)