nGraph-HE build not working

[pjattke]

See reported issue. If no response received till July 30, send a mail to Fabian Boemer (fabian.boemer@intel.com). Set Rosario Cammarota (rosario.cammarota@intel.com) and Casimir Wierzynski (casimir.wierzynski@intel.com) in CC?

Mail snippets:

We are currently working on a survey of FHE compilers & optimisation techniques for an SoK-style paper on the topic ...

We're planning to benchmark and compare as many tools & techniques as possible, including some apples-to-oranges comparison between tools with very different focuses. We want to explore what types of applications benefit most from which kinds of optimisations, aiding developers in selecting tools based on their application needs, and maybe identify opportunities for synergies between tools.

We would love to include in our evaluation, and .... *however,

<we would be delighted to hear ...>

[pjattke]

/remind me tomorrow at 1 PM about writing Fabian Boemer regarding nGraph-HE build issues.

[reminders[bot]]

@pjattke set a reminder for Jul 30th 2020

[pjattke]

Mail to Fabian sent (co-authors in CC) to ask for help in building nGraph-HE.

[reminders[bot]]

:wave: @pjattke, about writing Fabian Boemer regarding nGraph-HE build issues.

[pjattke]

Response received on July 30, 18:26:

Hi Patrick,

Thanks for your interest and efforts to use he-transformer. We do have CI enabled for past commits (e.g. https://github.com/IntelAI/he-transformer/pull/34 successfully built on CI). Unfortunately, I’m running into the same errors as you are, revolving around bfloat16.cc. Perhaps it may help to revert ngraph-tf to an earlier commit/version. I’m not sure I can offer any guidance beyond what you’ve already tried.

I’m currently not actively working on he-transformer, which unfortunately means issues like this one may go unnoticed/unresolved for some time.

Let me know if there’s anything else I can help with.

Best, Fabian

[AlexanderViand]

A case of but let's try that commit

[pjattke]

The mentioned commit does not work either. However, I could fix the error with bfloat16.cc which is related to numpy. I could fix it by forcing ngraph-bridge to use a version smaller than 1.19.0.

I now receive the following error by using the latest he-transformer commit on master:

...
#29 285.1 CMakeFiles/unit-test.dir/test_tcp_client.cpp.o: In function `operator()':
#29 285.1 /home/he-transformer/test/test_tcp_client.cpp:72: undefined reference to `ngraph::LogHelper::LogHelper(ngraph::LOG_TYPE, char const*, int, std::function<void (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>)'
#29 285.2 ../src/libhe_seal_backend.so: undefined reference to `ngraph::Node::get_provenance_tags[abi:cxx11]() const'
#29 285.3 ../src/libhe_seal_backend.so: undefined reference to `ngraph::file_util::read_file_to_string(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
#29 285.3 ../src/libhe_seal_backend.so: undefined reference to `ngraph::to_lower(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
#29 285.3 ../src/libhe_seal_backend.so: undefined reference to `ngraph::runtime::Backend::get_backend_op(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, ...)'
#29 285.3 ../src/libhe_seal_backend.so: undefined reference to `ngraph::runtime::BackendManager::register_backend(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::function<std::shared_ptr<ngraph::runtime::Backend> (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>)'
#29 285.3 ../src/libhe_seal_backend.so: undefined reference to `ngraph::file_util::exists(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
#29 285.3 ../src/libhe_seal_backend.so: undefined reference to `ngraph::descriptor::Tensor::get_name[abi:cxx11]() const'
#29 285.3 ../src/libhe_seal_backend.so: undefined reference to `ngraph::descriptor::Tensor::Tensor(ngraph::element::Type const&, ngraph::PartialShape const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
#29 285.3 ../src/libhe_seal_backend.so: undefined reference to `ngraph::split(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, char, bool)'
#29 285.3 ../src/libhe_seal_backend.so: undefined reference to `ngraph::Node::description[abi:cxx11]() const'
#29 285.3 clang: error: linker command failed with exit code 1 (use -v to see invocation)
#29 285.3 make[2]: *** [test/unit-test] Error 1
#29 285.3 test/CMakeFiles/unit-test.dir/build.make:825: recipe for target 'test/unit-test' failed
#29 285.3 make[2]: Leaving directory '/home/he-transformer/build'
#29 285.3 make[1]: *** [test/CMakeFiles/unit-test.dir/all] Error 2

See this gist for the full log (use the raw mode to see the whole file). I created a temporary repository to keep track of the build files used, see pjattke/docker-he-transformer.

@AlexanderViand I am not sure if I understand the log correctly but it looks like the library libhe_seal_backend.so that is built by he-transformer in this CMakeLists.txt does not link correctly against ngraph, right? Or do you think it could be because of wrong includes?

Note: This only looks like it is related to the tests but as I could not build + install the Python client bindings for he-transformer successfully, I believe this issue with libhe_seal_backend.so needs to be fixed to make it work.

[pjattke]

TODO

• [ ] Try using a prebuilt tensorflow to speed up build process and facilitate debugging the linker error.

I think this can be achieved by patching ngraph-tf.cmake by extending the build command with --use_prebuilt_tensorflow and installing tensorflow v1.14.0 in the Docker container prior starting the he-transformer build process.

[pjattke]

I sent a request to Fabian on Saturday asking if he knows this issue. Yesterday I received his response:

Hi Patrick, Glad to hear you were able to resolve the bfloat issue. This looks like an issue with the CXX ABI. I suspect you are compiling ngraph with an older version (< 5.1) of g++, but he-transformer with clang++-9. Indeed, I see several strings “compiler identification is GNU 4.8.5” in the build log at https://github.com/pjattke/docker-he-transformer/blob/master/docker-build-1596267947.log This looks like a bug, where he-transformer doesn’t pass the CMAKE_CXX_COMPILER to the external projects. For now, you should be able to install a more recent version of GCC and set it as the default. (i.e. g++ -v should be > 5.1). This will ensure ngraph is built with the CXX11 ABI, and will hopefully resolve this issues. Best, Fabian

I will try out his suggested solution.

[pjattke]

• [X] Fix build issue
• [X] Write short "Thank you" mail to Fabian
• [X] Update StackOverflow post
• [X] Add build files to SoK repository