Using C++ API : Not enough functions in libtensorflow_cc.so

[imnotkind]

Hello. I appreciate your work and I succeeded in building tensorflow 1.13.1 by your script. I just wanted to run an example on Visual Studio 2017, and I chose this one.

// tensorflow/cc/example/example.cc

#include "tensorflow/cc/client/client_session.h"
#include "tensorflow/cc/ops/standard_ops.h"
#include "tensorflow/core/framework/tensor.h"

int main() {
  using namespace tensorflow;
  using namespace tensorflow::ops;
  Scope root = Scope::NewRootScope();
  // Matrix A = [3 2; -1 0]
  auto A = Const(root, { {3.f, 2.f}, {-1.f, 0.f} });
  // Vector b = [3 5]
  auto b = Const(root, { {3.f, 5.f} });
  // v = Ab^T
  auto v = MatMul(root.WithOpName("v"), A, b, MatMul::TransposeB(true));
  std::vector<Tensor> outputs;
  ClientSession session(root);
  // Run and fetch v
  TF_CHECK_OK(session.Run({v}, &outputs));
  // Expect outputs[0] == [19; -3]
  LOG(INFO) << outputs[0].matrix<float>();
  return 0;
}

I included

source\bazel-source\external\protobuf_archive\src
source\bazel-source\external\com_google_absl
source\bazel-source\external\eigen_archive\
source\

to get rid of some third party related include errors. (protobuf, absl, eigen) I guess I achieved that goal, since I got rid of include errors.

and I only set tensorflow_cc.lib and tensorflow_cc.dll as for any static/dynamic libraries.

At the first try, I had some lack of symbols, and I used dumpbin to extract every symbol from tensorflow_cc.dll, and made my own tensorflow_cc.lib. That got rid of some unresolved external symbol errors, but not all of them. The thing is, this tensorflow_cc.dll we compiled by bazel just doesn't have enough functions. (Or that's what I think so) if you look at https://joe-antognini.github.io/machine-learning/windows-tf-project, (which is an example of using cmake to integrate tensorflow in windows, but cmake is broke for recent versions, I tried a lot.) he sets a tremendous amount of linker settings

zlib\install\lib\zlibstatic.lib
gif\install\lib\giflib.lib
png\install\lib\libpng12_static.lib
jpeg\install\lib\libjpeg.lib
lmdb\install\lib\lmdb.lib
jsoncpp\src\jsoncpp\src\lib_json\$(Configuration)\jsoncpp.lib
farmhash\install\lib\farmhash.lib
fft2d\\src\lib\fft2d.lib
highwayhash\install\lib\highwayhash.lib
libprotobuf.lib
tf_protos_cc.lib
tf_cc.lib
tf_cc_ops.lib
tf_cc_framework.lib
tf_core_cpu.lib
tf_core_direct_session.lib
tf_core_framework.lib
tf_core_kernels.lib
tf_core_lib.lib
tf_core_ops.lib

and I'm highly convinced that our single tensorflow_cc.dll doesn't have all the functions in that amount of libs.

I assume you guys are able to actually test and run the program with C++ api. Can you please tell me the details? What did you include? What static/dynamic libraries did you use? What is the code you used for testing? And last, what should I do to find my needed functions, which are

1>example.obj : error LNK2001: "public: __cdecl tensorflow::ops::MatMul::MatMul(class tensorflow::Scope const &,class tensorflow::Input,class tensorflow::Input,struct tensorflow::ops::MatMul::Attrs const &)" (??0MatMul@ops@tensorflow@@QEAA@AEBVScope@2@VInput@2@1AEBUAttrs@012@@Z) 
1>example.obj : error LNK2001: "class tensorflow::Output __cdecl tensorflow::ops::Const(class tensorflow::Scope const &,struct tensorflow::Input::Initializer const &)" (?Const@ops@tensorflow@@YA?AVOutput@2@AEBVScope@2@AEBUInitializer@Input@2@@Z) 
1>example.obj : error LNK2001: "public: virtual __cdecl tensorflow::internal::LogMessage::~LogMessage(void)" (??1LogMessage@internal@tensorflow@@UEAA@XZ) 
1>example.obj : error LNK2001: "public: __cdecl tensorflow::internal::LogMessage::LogMessage(char const *,int,int)" (??0LogMessage@internal@tensorflow@@QEAA@PEBDHH@Z) 
1>example.obj : error LNK2001: "private: class tensorflow::Scope __cdecl tensorflow::Scope::WithOpNameImpl(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const &)const " (?WithOpNameImpl@Scope@tensorflow@@AEBA?AV12@AEBV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@Z) 
1>example.obj : error LNK2001: "public: static class tensorflow::Scope __cdecl tensorflow::Scope::NewRootScope(void)" (?NewRootScope@Scope@tensorflow@@SA?AV12@XZ) 
1>example.obj : error LNK2001: "public: __cdecl tensorflow::Scope::~Scope(void)" (??1Scope@tensorflow@@QEAA@XZ) 
1>example.obj : error LNK2001: "public: class tensorflow::Status __cdecl tensorflow::ClientSession::Run(class std::vector<class tensorflow::Output,class std::allocator<class tensorflow::Output> > const &,class std::vector<class tensorflow::Tensor,class std::allocator<class tensorflow::Tensor> > *)const " (?Run@ClientSession@tensorflow@@QEBA?AVStatus@2@AEBV?$vector@VOutput@tensorflow@@V?$allocator@VOutput@tensorflow@@@std@@@std@@PEAV?$vector@VTensor@tensorflow@@V?$allocator@VTensor@tensorflow@@@std@@@5@@Z) 
1>example.obj : error LNK2001: "public: __cdecl tensorflow::ClientSession::~ClientSession(void)" (??1ClientSession@tensorflow@@QEAA@XZ) 
1>example.obj : error LNK2001: "public: __cdecl tensorflow::ClientSession::ClientSession(class tensorflow::Scope const &)" (??0ClientSession@tensorflow@@QEAA@AEBVScope@1@@Z) 
1>example.obj : error LNK2001: "public: __cdecl tensorflow::Input::Initializer::Initializer(class std::initializer_list<struct tensorflow::Input::Initializer> const &)" (??0Initializer@Input@tensorflow@@QEAA@AEBV?$initializer_list@UInitializer@Input@tensorflow@@@std@@@Z)

in this example code?

[StevenRoodhorst]

The DLL is built without exposing any symbols, so any symbol you want to have, you will have to add it yourself. Please check this reply

I myself wrote a FindTensorFlow.cmake file, which you can use with a find_package(TensorFlow REQUIRED) call in your cmake project . That shows a bit how to link all the libs and how to add the include dirs.

find_package(TensorFlow 1.11.0 REQUIRED)
target_include_directories(<your-target> ${TensorFlow_INCLUDE_DIRS})
target_link_libraries(<your-target> ${TensorFlow_LIBRARIES})

FindTensorFlow.cmake:

# Locates the TensorFlow v1.11.0 C++ shared library and include directories.

include(FindPackageHandleStandardArgs)
unset(TENSORFLOW_FOUND)

message(STATUS "TensorFlow_DIR is defined as ${TensorFlow_DIR}")

# Finding main TensorFlow library.
find_library(TensorFlow_LIBRARY 
    NAMES
        tensorflow_cc
        libtensorflow_cc.so.if
        libtensorflow_cc.so
    PATHS 
        ${TensorFlow_DIR}/tensorflow/lib
)

# Finding TensorFlow framework library if available. 
# Only needed in case the framework library is built together with the cc library.
find_library(TensorFlow_FW_LIBRARY 
    NAMES 
        tensorflow_framework
        libtensorflow_framework.so
    PATHS 
        ${TensorFlow_DIR}/tensorflow/lib
)

# Finding TensorFlow main include directory.
find_path(TensorFlow_INCLUDE_DIR
    NAMES
        tensorflow
    PATHS
        ${TensorFlow_DIR}/tensorflow/include
)

# Finding TensorFlow protobuf include directory.
# On Linux this is merged with the main include directory.
if (WIN32)
    find_path(TensorFlow_INCLUDE_DIR_pb
        NAMES
            tensorflow
        PATHS
            ${TensorFlow_DIR}/tensorflow/include_pb
    )
endif()

# Finding Eigen include directory.
find_path(Eigen_INCLUDE_DIR_third_party
    NAMES
        third_party
    PATHS
        ${TensorFlow_DIR}/Eigen/include
)

# Finding Abseil include directory.
find_path(ABSL_INCLUDE_DIR
    NAMES
        absl
    PATHS
        ${TensorFlow_DIR}/absl/include
)

# Find Protobuf package.
option(protobuf_MODULE_COMPATIBLE BOOL ON)
find_package(Protobuf 3.6.0 REQUIRED NO_DEFAULT_PATH 
    PATHS
        ${TensorFlow_DIR}/protobuf/lib/cmake/protobuf/
        ${TensorFlow_DIR}/protobuf/lib64/cmake/protobuf/
        ${TensorFlow_DIR}/protobuf/cmake/
)

# Find Eigen3 package.
message(STATUS "Finding Eigen...")
set(ENV{EIGEN3_ROOT_DIR} "${TensorFlow_DIR}/Eigen/eigen_archive")
find_package(Eigen3 QUIET
    PATHS
        ${TensorFlow_DIR}/Eigen/eigen_archive/cmake
)

if(NOT Eigen_FOUND)
  include(${TensorFlow_DIR}/Eigen/eigen_archive/cmake/FindEigen3.cmake)
  set(Eigen_INCLUDE_DIR ${EIGEN3_INCLUDE_DIR})
endif()

# set TensorFlow_FOUND
find_package_handle_standard_args(TensorFlow DEFAULT_MSG TensorFlow_INCLUDE_DIR TensorFlow_LIBRARY)

# Set external variables for usage in CMakeLists.txt
if(TENSORFLOW_FOUND)
    set(TensorFlow_LIBRARIES 
        "${TensorFlow_LIBRARY}"
        "${Protobuf_LIBRARIES}"
         "Eigen3::Eigen"
    )

    # Add TensorFlow framework library if found.
    if(TensorFlow_FW_LIBRARY)
        list(APPEND TensorFlow_LIBRARIES "${TensorFlow_FW_LIBRARY}")
    endif()

    set(TensorFlow_INCLUDE_DIRS 
        "${TensorFlow_INCLUDE_DIR}"
        "${Eigen_INCLUDE_DIR_third_party}"
        "${ABSL_INCLUDE_DIR}"
        "${Protobuf_INCLUDE_DIRS}"
    )

    if (WIN32)
        list(APPEND TensorFlow_INCLUDE_DIRS "${TensorFlow_INCLUDE_DIR_pb}")
    endif()
endif()

# Hide local variables from GUI.
mark_as_advanced(
    Protobuf_LIBRARIES
    TensorFlow_LIBRARY
    TensorFlow_FW_LIBRARY
    TensorFlow_INCLUDE_DIR
    TensorFlow_INCLUDE_DIR_pb
    Protobuf_INCLUDE_DIRS
    Eigen_INCLUDE_DIR
    Eigen_INCLUDE_DIR_third_party
    ABSL_INCLUDE_DIR
)

The files used are copied with the commands below:

# Install tensorflow lib and includes to bin.
mkdir $TensorFlowBinDir\tensorflow\lib\ -ErrorAction SilentlyContinue
Copy-Item  $TensorFlowSourceDir\bazel-bin\tensorflow\libtensorflow_cc.so $TensorFlowBinDir\tensorflow\lib\tensorflow_cc.dll -Force
Copy-Item  $TensorFlowSourceDir\bazel-bin\tensorflow\libtensorflow_cc.so.if.lib $TensorFlowBinDir\tensorflow\lib\tensorflow_cc.lib -Force

Copy-Item $TensorFlowSourceDir\tensorflow\core $TensorFlowBinDir\tensorflow\include\tensorflow\core -Recurse -Container  -Filter "*.h" -Force
Copy-Item $TensorFlowSourceDir\tensorflow\cc $TensorFlowBinDir\tensorflow\include\tensorflow\cc -Recurse -Container -Filter "*.h" -Force

Copy-Item $TensorFlowSourceDir\bazel-genfiles\tensorflow\core\ $TensorFlowBinDir\tensorflow\include_pb\tensorflow\core -Recurse -Container -Filter "*.h" -Force
Copy-Item $TensorFlowSourceDir\bazel-genfiles\tensorflow\cc $TensorFlowBinDir\tensorflow\include_pb\tensorflow\cc -Recurse -Container -Filter "*.h" -Force

# Absl includes.
Copy-Item $TensorFlowSourceDir\bazel-source\external\com_google_absl\absl $TensorFlowBinDir\absl\include\absl\ -Recurse -Container -Filter "*.h" -Force

# Eigen includes
Copy-Item $TensorFlowSourceDir\bazel-source\external\eigen_archive\ $TensorFlowBinDir\Eigen\eigen_archive -Recurse -Force
Copy-Item $TensorFlowSourceDir\third_party\eigen3 $TensorFlowBinDir\Eigen\include\third_party\eigen3\ -Recurse -Force

[imnotkind]

Sorry for not trying to build once again. I built it again with my symbols in tf_exported_symbols_msvc.lds, and it worked! thanks :) I'm not used to Cmake files, but I probably assume that the copied files from below are the required header and library files, and Cmake is used to generate a project that has those dependencies? I get most of the things, but how did you install protobuf? I just included source\bazel-source\external\protobuf_archive\src, but it seems you have a different include path with me.

# Find Protobuf package.
option(protobuf_MODULE_COMPATIBLE BOOL ON)
find_package(Protobuf 3.6.0 REQUIRED NO_DEFAULT_PATH 
    PATHS
        ${TensorFlow_DIR}/protobuf/lib/cmake/protobuf/
        ${TensorFlow_DIR}/protobuf/lib64/cmake/protobuf/
        ${TensorFlow_DIR}/protobuf/cmake/
)

[imnotkind]

And what about this?

# Finding TensorFlow framework library if available. 
# Only needed in case the framework library is built together with the cc library.
find_library(TensorFlow_FW_LIBRARY 
    NAMES 
        tensorflow_framework
        libtensorflow_framework.so
    PATHS 
        ${TensorFlow_DIR}/tensorflow/lib
)

Is it ok for me to ignore this library, when libtensorflow_framework.so was NOT in my bazel-bin as output of bazel build //tensorflow:libtensorflow_cc.so, or do I need to compile that library additionally?

[StevenRoodhorst]

Sorry for not trying to build once again. I built it again with my symbols in tf_exported_symbols_msvc.lds, and it worked! thanks :) I'm not used to Cmake files, but I probably assume that the copied files from below are the required header and library files, and Cmake is used to generate a project that has those dependencies? I get most of the things, but how did you install protobuf? I just included source\bazel-source\external\protobuf_archive\src, but it seems you have a different include path with me.

# Find Protobuf package.
option(protobuf_MODULE_COMPATIBLE BOOL ON)
find_package(Protobuf 3.6.0 REQUIRED NO_DEFAULT_PATH 
    PATHS
        ${TensorFlow_DIR}/protobuf/lib/cmake/protobuf/
        ${TensorFlow_DIR}/protobuf/lib64/cmake/protobuf/
        ${TensorFlow_DIR}/protobuf/cmake/
)

Hi, I build my own protobuf CMake project and include the headers and library.

Second question: Indeed you do not need that TensorFlow framework library. We had some systems on which it was compiled additionally. If you don't have it and it works, then leave it out.

[terryzhao127]

@Steroes Does it mean building protobuf using CMake is still needed?

[imnotkind]

well for my case, just including source\bazel-source\external\protobuf_archive\src without the need of any .lib files worked, at least for the c++ example, but I'm not sure if it's because I just ran a tutorial code.

[terryzhao127]

@imnotkind Can you run the built binary without linking *.lib files?

[StevenRoodhorst]

@Steroes Does it mean building protobuf using CMake is still needed?

I still need it, but I don't use the configuration from this repository. Not sure if it is still needed when the files are included that @imnotkind is referring to.

[imnotkind]

well, I think it's not needed if you include those files. thank you guys for your help, and for someone in need, I'm going to leave my extraction script. https://gist.github.com/imnotkind/6e41217ee8e6cf1373a883681d245029