Problems with build on macOS Sierra

[MyOwnClone]

Hi, thanks for great project, awesome work. I've had some problems trying to build tf-coriander from sources on macOS Sierra. At first, I want to point out that maybe I've misunderstood the instructions in the https://github.com/hughperkins/tf-coriander/blob/master/doc/build-from-source.md article, or maybe I've just missed something and I am doing something completely wrong. In this case, I apologize.

If I understand correctly, according to mentioned page, I normally clone the repository, go to it and call bash ./install_deps.sh. This worked, or seemed to work. Than, I've called bash ./build.sh from the same folder. This is output from repeated call, so something is expected to be already up to date:

+ . env3/bin/activate
++ deactivate nondestructive
++ '[' -n '' ']'
++ '[' -n '' ']'
++ '[' -n /bin/bash -o -n '' ']'
++ hash -r
++ '[' -n '' ']'
++ unset VIRTUAL_ENV
++ '[' '!' nondestructive = nondestructive ']'
++ VIRTUAL_ENV=/Users/tomasv/work/github/tf-coriander/env3
++ export VIRTUAL_ENV
++ _OLD_VIRTUAL_PATH=/Users/tomasv/work/github/tf-coriander/env3/bin:/Users/tomasv/coriander/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/share/dotnet:/usr/local/go/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands
++ PATH=/Users/tomasv/work/github/tf-coriander/env3/bin:/Users/tomasv/work/github/tf-coriander/env3/bin:/Users/tomasv/coriander/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/share/dotnet:/usr/local/go/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands
++ export PATH
++ '[' -n '' ']'
++ '[' -z '' ']'
++ _OLD_VIRTUAL_PS1=
++ '[' 'x(env3) ' '!=' x ']'
++ PS1='(env3) '
++ export PS1
++ '[' -n /bin/bash -o -n '' ']'
++ hash -r
+ CLANG_HOME=/Users/tomasv/work/github/tf-coriander/soft/llvm-4.0
+ util/build_coriander.sh
+ git submodule update --init --recursive
+ pushd third_party/coriander
~/work/github/tf-coriander/third_party/coriander ~/work/github/tf-coriander
+ [[ ! -d build ]]
+ cd build
+ [[ x/Users/tomasv/work/github/tf-coriander/soft/llvm-4.0 != x ]]
+ cmake -DCMAKE_BUILD_TYPE=Debug -DCLANG_HOME=/Users/tomasv/work/github/tf-coriander/soft/llvm-4.0 ..
-- Configuring done
CMake Warning (dev):
  Policy CMP0042 is not set: MACOSX_RPATH is enabled by default.  Run "cmake
  --help-policy CMP0042" for policy details.  Use the cmake_policy command to
  set the policy and suppress this warning.

  MACOSX_RPATH is not specified for the following targets:

   clblast
   clew
   cocl
   cocl_gtest
   easycl

This warning is for project developers.  Use -Wno-dev to suppress it.

-- Generating done
-- Build files have been written to: /Users/tomasv/work/github/tf-coriander/third_party/coriander/build
+ make -j 8
[  1%] Built target clew
[  9%] Built target patch_hostside
[ 15%] Built target easycl
[ 50%] Built target clblast
[ 96%] Built target cocl
[100%] Built target ir-to-opencl
+ SUDO=sudo
++ cat /proc/1/sched
++ head -n 1
++ grep '[init|systemd]'
cat: /proc/1/sched: No such file or directory
+ [[ ! -n '' ]]
+ echo running in docker
running in docker
+ SUDO=
++ uname
+ [[ Darwin == Darwin ]]
+ SUDO=
+ make install
[  1%] Built target clew
[ 35%] Built target clblast
[ 42%] Built target easycl
[ 89%] Built target cocl
[ 92%] Built target ir-to-opencl
[100%] Built target patch_hostside
Install the project...
-- Install configuration: "Debug"
-- Up-to-date: /usr/local/include/cocl/__clang_cuda_runtime_wrapper.h
-- Up-to-date: /usr/local/include/cocl/cocl.h
-- Up-to-date: /usr/local/include/cocl/cocl_blas.h
-- Up-to-date: /usr/local/include/cocl/cocl_clsources.h
-- Up-to-date: /usr/local/include/cocl/cocl_context.h
-- Up-to-date: /usr/local/include/cocl/cocl_defs.h
-- Up-to-date: /usr/local/include/cocl/cocl_device.h
-- Up-to-date: /usr/local/include/cocl/cocl_deviceside.h
-- Up-to-date: /usr/local/include/cocl/cocl_dnn.h
-- Up-to-date: /usr/local/include/cocl/cocl_dnn_act.h
-- Up-to-date: /usr/local/include/cocl/cocl_dnn_conv.h
-- Up-to-date: /usr/local/include/cocl/cocl_dnn_core.h
-- Up-to-date: /usr/local/include/cocl/cocl_dnn_gemm.h
-- Up-to-date: /usr/local/include/cocl/cocl_dnn_pooling.h
-- Up-to-date: /usr/local/include/cocl/cocl_error.h
-- Up-to-date: /usr/local/include/cocl/cocl_events.h
-- Up-to-date: /usr/local/include/cocl/cocl_hostside.h
-- Up-to-date: /usr/local/include/cocl/cocl_kernellaunch.h
-- Up-to-date: /usr/local/include/cocl/cocl_memory.h
-- Up-to-date: /usr/local/include/cocl/cocl_properties.h
-- Up-to-date: /usr/local/include/cocl/cocl_streams.h
-- Up-to-date: /usr/local/include/cocl/cublas_v2.h
-- Up-to-date: /usr/local/include/cocl/cuda.h
-- Up-to-date: /usr/local/include/cocl/cuda_config.h
-- Up-to-date: /usr/local/include/cocl/cuda_runtime_api.h
-- Up-to-date: /usr/local/include/cocl/cudnn.h
-- Up-to-date: /usr/local/include/cocl/fake_funcs.h
-- Up-to-date: /usr/local/include/cocl/hostside_opencl_funcs.h
-- Up-to-date: /usr/local/include/cocl/local_config.h.templ
-- Up-to-date: /usr/local/include/cocl/math_constants.h
-- Up-to-date: /usr/local/include/cocl/cuda_device_runtime_api.h
-- Up-to-date: /usr/local/include/cocl/cuda_runtime.h
-- Up-to-date: /usr/local/include/cocl/device_launch_parameters.h
-- Up-to-date: /usr/local/include/cocl/driver_types.h
-- Up-to-date: /usr/local/include/cocl/math_functions.hpp
-- Up-to-date: /usr/local/include/cocl/vector_types.h
-- Up-to-date: /usr/local/bin/cocl_wrapped
-- Up-to-date: /usr/local/bin/cocl
-- Up-to-date: /usr/local/include/clew.h
-- Up-to-date: /usr/local/include/EasyCL/CLArray.h
-- Up-to-date: /usr/local/include/EasyCL/CLArrayFloat.h
-- Up-to-date: /usr/local/include/EasyCL/CLArrayInt.h
-- Up-to-date: /usr/local/include/EasyCL/CLFloatWrapper.h
-- Up-to-date: /usr/local/include/EasyCL/CLFloatWrapperConst.h
-- Up-to-date: /usr/local/include/EasyCL/CLIntWrapper.h
-- Up-to-date: /usr/local/include/EasyCL/CLKernel.h
-- Up-to-date: /usr/local/include/EasyCL/CLKernel_structs.h
-- Up-to-date: /usr/local/include/EasyCL/CLUCharWrapper.h
-- Up-to-date: /usr/local/include/EasyCL/CLWrapper.h
-- Up-to-date: /usr/local/include/EasyCL/DeviceInfo.h
-- Up-to-date: /usr/local/include/EasyCL/DevicesInfo.h
-- Up-to-date: /usr/local/include/EasyCL/EasyCL.h
-- Up-to-date: /usr/local/include/EasyCL/EasyCL_export.h
-- Up-to-date: /usr/local/include/EasyCL/ICLArray.h
-- Up-to-date: /usr/local/include/EasyCL/deprecated.h
-- Up-to-date: /usr/local/include/EasyCL/deviceinfo_helper.h
-- Up-to-date: /usr/local/include/EasyCL/mystdint.h
-- Up-to-date: /usr/local/include/EasyCL/platforminfo_helper.h
-- Up-to-date: /usr/local/share/cocl/cocl.cmake
-- Up-to-date: /usr/local/share/cocl/cocl_impl.cmake
-- Installing: /usr/local/lib/libeasycl.dylib
-- Installing: /usr/local/lib/libclew.dylib
-- Installing: /usr/local/lib/libclblast.dylib
-- Installing: /usr/local/lib/libcocl.dylib
-- Up-to-date: /usr/local/bin/patch_hostside
-- Up-to-date: /usr/local/lib/cocl/cocl-targets.cmake
-- Installing: /usr/local/lib/cocl/cocl-targets-debug.cmake
+ popd
~/work/github/tf-coriander
+ echo Installed coriander
Installed coriander
+ util/run_configure.sh
+ cat
+ ./configure
~/work/github/tf-coriander ~/work/github/tf-coriander
No Google Cloud Platform support will be enabled for TensorFlow
No Hadoop File System support will be enabled for TensorFlow

After that, according to instruction page, there should be whl wheel in soft/tensorflowpkg folder. But these instructions are IMHO wrong, because no TF build have taken place. This is confirmed by the absence of the tensorflowpkg subfolder in soft folder and also the whl file is of course not present. But as I remember, TF must be built by calling bazel, so I have tried to build TF by using:

bazel build --jobs 4 //tensorflow/tools/pip_package:build_pip_package

This yielded a lot of errors, saying that set() syntax is deprecated and I should use depset(). I have replaced all set() calls with depset() calls in main bzl file and re-run command. This is what I got:

DEBUG: /private/var/tmp/_bazel_tomasv/f8bf8395d5f33d0a7fe51eea33d94dd9/external/bazel_tools/tools/osx/xcode_configure.bzl:85:5: Invoking xcodebuild failed, developer dir: /Applications/Xcode.app/Contents/Developer ,return code 256, stderr: Process terminated by signal 15, stdout:
ERROR: /Users/tomasv/work/github/tf-coriander/tensorflow/python/BUILD:1667:1: Traceback (most recent call last):
    File "/Users/tomasv/work/github/tf-coriander/tensorflow/python/BUILD", line 1667
        tf_proto_library_py(name = "compare_test_proto", testo..., ..."])
    File "/Users/tomasv/work/github/tf-coriander/tensorflow/core/platform/default/build_config.bzl", line 62, in tf_proto_library_py
        py_proto_library(name = (name + "_py"), srcs = srcs, <6 more arguments>)
    File "/private/var/tmp/_bazel_tomasv/f8bf8395d5f33d0a7fe51eea33d94dd9/external/protobuf/protobuf.bzl", line 335, in py_proto_library
        py_libs += [default_runtime]
trying to mutate a frozen object
ERROR: /Users/tomasv/work/github/tf-coriander/tensorflow/python/BUILD:231:1: Target '//tensorflow/python:python_op_gen' contains an error and its package is in error and referenced by '//tensorflow/python:python_op_gen_main'
ERROR: /Users/tomasv/work/github/tf-coriander/tensorflow/python/BUILD:231:1: Target '//tensorflow/python:framework/python_op_gen_main.cc' contains an error and its package is in error and referenced by '//tensorflow/python:python_op_gen_main'
ERROR: /Users/tomasv/work/github/tf-coriander/tensorflow/models/embedding/BUILD:107:1: Target '//tensorflow/python:python_op_gen_main' contains an error and its package is in error and referenced by '//tensorflow/models/embedding:gen_gen_word2vec_py_wrappers_cc'
ERROR: /Users/tomasv/work/github/tf-coriander/tensorflow/models/embedding/BUILD:107:1: Target '//tensorflow/python:framework_for_generated_wrappers' contains an error and its package is in error and referenced by '//tensorflow/models/embedding:gen_word2vec'
ERROR: /Users/tomasv/work/github/tf-coriander/tensorflow/tools/pip_package/BUILD:23:1: Target '//tensorflow/python:util_example_parser_configuration' contains an error and its package is in error and referenced by '//tensorflow/tools/pip_package:build_pip_package'
ERROR: Analysis of target '//tensorflow/tools/pip_package:build_pip_package' failed; build aborted: Loading failed
INFO: Elapsed time: 1,946s
FAILED: Build did NOT complete successfully (7 packages loaded)

I think that most important is the trying to mutate a frozen object part...But I do not know how to fix the build in this stage. Am I doing it absolutely wrong or did I understand something wrong? Is documentation correct?

Thanks for the response, Tom

[acrogenesis]

I'm doing the same as you did but after replacing set() to depset() I got this different error:

ERROR: /Users/acrogenesis/Downloads/tf-coriander/WORKSPACE:23:1: Traceback (most recent call last):
    File "/Users/acrogenesis/Downloads/tf-coriander/WORKSPACE", line 23
        check_version("0.3.0")
    File "/Users/acrogenesis/Downloads/tf-coriander/tensorflow/tensorflow.bzl", line 35, in check_version
        fail("\nCurrent Bazel version is {}, ...))

Current Bazel version is 0.11.1-homebrew, expected at least 0.3.0
ERROR: Error evaluating WORKSPACE file
ERROR: Skipping '@grpc//:grpc_cpp_plugin': error loading package 'external': Package 'external' contains errors
WARNING: Target pattern parsing failed.
ERROR: error loading package 'external': Package 'external' contains errors
INFO: Elapsed time: 0.248s
FAILED: Build did NOT complete successfully (0 packages loaded)

[acrogenesis]

@MyOwnClone could you finish the installation?

[bcaldir]

Is there any update on this issue? I'm facing with the exactly same problem of not having the whl file.

[KendallPark]

I am having this exact issue. No whl file to be found.

[rayryeng]

Same problem here. whl file is not produced. Building on Mac OS 10.13.6 (High Sierra).

[hughperkins]

Hi, i expect that the version of bazel has changes since the instructions were written and is incompatible. I think the easiest way forward would be to figure out which version of bazel was available at the time of the instructions, and explicitly use that version.

On Sun, Oct 14, 2018, 17:51 Raymond Phan notifications@github.com wrote:

Same problem here. whl file is not produced. Building on Mac OS 10.13.6 (Mojave).

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/hughperkins/tf-coriander/issues/82#issuecomment-429666335, or mute the thread https://github.com/notifications/unsubscribe-auth/AAHiqIwjwOwN-0KxPcLr0uvH33Kjg6mwks5uk7HmgaJpZM4STnCq .

[datatalking]

Looking for similar solution, think I sort of found two possible avenues of solution i'm going to test this weekend. @acrogenesis, @MyOwnClone @bcaldir, @rayryeng, @KendallPark, @hughperkins

[rayryeng]

@datatalking Thanks for this, but I've moved onto a Macbook with a M1 chip and there's already an established pipeline for getting TF and it uses the M1 GPU. I don't think I'll need (or am able) to test things out for you as I don't need the OpenCL flavour anymore.

[datatalking]

Thanks for replying, I was able to get my nvidia GPU working after rebuilding the whl.

On Jan 3, 2023, at 5:25 PM, Ray Phan @.***> wrote:

@datatalking https://github.com/datatalking Thanks for this, but I've moved onto a Macbook with a M1 chip and there's already an established pipeline for getting TF. I don't think I'll need (or am able) to test things out for you.

— Reply to this email directly, view it on GitHub https://github.com/hughperkins/tf-coriander/issues/82#issuecomment-1370389772, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC2ECWL3KR3LSO7E4CIQ62LWQTGRHANCNFSM4ESOOCVA. You are receiving this because you were mentioned.