CMake: FindTensorflow.cmake not working on MacOS

MiguelAlgaba commented 5 years ago

I was trying to compile some of the examples requiring the Tensorflow C-API, however so far I'm unable to get FindTensorflow.cmake to find the version of Tensorflow I installed from source

I'm using macOS High Sierra v10.13.6 I installed Tensorflow using the following commands:

TENSORFLOW_SOURCE_DIR=tensorflow cd ${TENSORFLOW_SOURCE_DIR} (latest master) ./configure (defaults settings on OSX) bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package (cpu-only build) ./bazel-bin/tensorflow/tools/pip_package/build_pip_package --nightly_flag /tmp/tensorflow_pkg pip install --ignore-installed wrapt /tmp/tensorflow_pkg/tf_nightly-1.13.1-cp36-cp36m-macosx_10_13_x86_64.whl

cd .. export TENSORFLOW_BUILD_DIR=tensorflow-build mkdir ${TENSORFLOW_BUILD_DIR} cp ${TENSORFLOW_SOURCE_DIR}/bazel-bin/tensorflow/*.so ${TENSORFLOW_BUILD_DIR}/ user@host $ cp ${TENSORFLOW_SOURCE_DIR}/bazel-genfiles/tensorflow/cc/ops/*.h ${TENSORFLOW_BUILD_DIR}/includes/tensorflow/cc/ops/ Then I tried example inside tensorflow-cmake/inference/cc

cmake .

And I get the following output

CMake Warning at /Users/miguel/Development/tensorflow-cmake/cmake/modules/FindTensorFlow.cmake:44 (message):
  This FindTensorflow.cmake is not tested on APPLE

  Please report if this works

  https://github.com/PatWie/tensorflow-cmake
Call Stack (most recent call first):
  CMakeLists.txt:6 (find_package)

-- Detecting TensorFlow using python3 (use -DPYTHON_EXECUTABLE=... otherwise)
-- Detecting TensorFlow info - done
CMake Error at /Users/miguel/Development/tensorflow-cmake/cmake/modules/FindTensorFlow.cmake:166 (message):
  Your installed TensorFlow version 1.13 is not supported

  We tested against 1.9;1.9.0;1.10;1.10.0;1.11;1.11.0;1.12;1.12.0
Call Stack (most recent call first):
  CMakeLists.txt:6 (find_package)

-- Configuring incomplete, errors occurred!
See also "/Users/miguel/Development/tensorflow-cmake/inference/cc/CMakeFiles/CMakeOutput.log".

Am I missing something?

Thanks a lot in advance!

PatWie commented 5 years ago

Is there really TensorFlow version "1.13"? I thought there is only "1.13.1". You might want to change the line

https://github.com/PatWie/tensorflow-cmake/blob/dc0b4eeee274849abc1effdb29df6b3cd5f39d06/cmake/modules/FindTensorFlow.cmake#L126

and add your version. As soon as I have the CI for building the ops under 1.13.1 I will add it here to the repository. However, there are so many things which have been changed.

I do not have any way to test this under any Apple-Device. Would be nice to make it work under MacOS.

MiguelAlgaba commented 5 years ago

Hi @PatWie thanks for your quick reply.

Adding 1.13 to the list of supported Tensorflow versions as you suggested and manually specifying the paths to the Tensorflow source and libraries I managed to get CMake to generate the makefile for the inference/cc example

However when I trying to build the example it says it cannot find <tensorflow/core/protobuf/meta_graph.pb.h>. Any ideas?

[ 50%] Building CXX object CMakeFiles/inference_cc.dir/inference_cc.cc.o
/Users/miguel/Development/tensorflow-cmake/inference/cc/inference_cc.cc:2:10: fatal error: 'tensorflow/core/protobuf/meta_graph.pb.h' file not found
#include <tensorflow/core/protobuf/meta_graph.pb.h>
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
make[2]: *** [CMakeFiles/inference_cc.dir/inference_cc.cc.o] Error 1
make[1]: *** [CMakeFiles/inference_cc.dir/all] Error 2
make: *** [all] Error 2

Thanks in advance!

MiguelAlgaba commented 5 years ago

Copying all the content of tensorflow/bazel-genfiles/tensorflow to tensorflow-build/includes manages to find tensorflow/core/protobuf/meta_graph.pb.h although it fails to build too

Scanning dependencies of target inference_cc
[ 50%] Building CXX object CMakeFiles/inference_cc.dir/inference_cc.cc.o
In file included from /Users/miguel/Development/tensorflow-cmake/inference/cc/inference_cc.cc:2:
/Users/miguel/Development/tensorflow-build/includes/tensorflow/core/protobuf/meta_graph.pb.h:10:10: fatal error: 'google/protobuf/port_def.inc' file not found
#include <google/protobuf/port_def.inc>
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
make[2]: *** [CMakeFiles/inference_cc.dir/inference_cc.cc.o] Error 1
make[1]: *** [CMakeFiles/inference_cc.dir/all] Error 2
make: *** [all] Error 2

PatWie commented 5 years ago

This is strange as bazel downloads all dependencies including Google protobuf. Installing the protobuf by hand is a bad idea as version missmatchs can add additional issues.

Can you confirm there is no such file "port_def.inc". Which tensor flow version do you issue. It can be that they broke TF again:

https://github.com/tensorflow/tensorflow/issues/27697

MiguelAlgaba commented 5 years ago

Hi @PatWie yes I was using a more recent Tensorflow version from commit f90532431c3785166cff35ff427b652fe460f60b, so seems like the "port_def.inc" issue is related to the tensorflow/tensorflow#27697 issue you posted indeed.

I decided to use stable release v1.13.1 and compiling that version from source almost works as you can see here. Now the remaining error seems to be a linking issue

MiguelAlgaba commented 5 years ago

For reference this is how the paths look when calling cmake. They are automatically found when exporting the paths to TENSORFLOW_BUILD_DIR and TENSORFLOW_SOURCE_DIR

MiguelAlgaba commented 5 years ago

Locally removing the unknown options --allow-multiple-definition -Wl,--whole-archive ${TensorFlow_LIBRARY} -Wl,--no-whole-archive I get undefined symbols

macos_undefined_symbols.txt

PatWie commented 5 years ago

These flags are important indeed (for g++). MacOS uses llvm. Not sure if an equivalent exists:(https://stackoverflow.com/a/17445806/7443104)

PatWie / tensorflow-cmake

CMake: FindTensorflow.cmake not working on MacOS #39