search

intel / he-transformer

nGraph-HE: Deep learning with Homomorphic Encryption (HE) through Intel nGraph
Apache License 2.0
172 stars 35 forks source link

Build Docker with ABY #51

Open adam-dziedzic opened 4 years ago

adam-dziedzic commented 4 years ago

To build ABY, g++ version >= 8.4 is required. However, by default, the docker version of Ubuntu 18.04 comes with g++ version 7.5 (as of now). Thus, we should add g++8 (install g++8) to the build of the docker image.

Moreover, I encountered a few problems while building the library inside docker. The main issue was that it seemed as if not the whole boost library was built and then ABY failed to build. I built the boost library fully manually and then was able to build ABY. Finally, I ran the tests for ABY and all passed. The basic python example examples/ax.py also worked.

adam-dziedzic commented 4 years ago

The boost library is not fully build after: make install

ENCRYPTO_utils was not found: add ENCRYPTO_utils subdirectory
CMake Error at /usr/local/lib/python3.6/dist-packages/cmake/data/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
  **Could NOT find Boost (missing: system thread) (found suitable version
  "1.69.0", minimum required is "1.66.0")**
Call Stack (most recent call first):
  /usr/local/lib/python3.6/dist-packages/cmake/data/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:458 (_FPHSA_FAILURE_MESSAGE)
  /usr/local/lib/python3.6/dist-packages/cmake/data/share/cmake-3.18/Modules/FindBoost.cmake:2177 (find_package_handle_standard_args)  extern/ENCRYPTO_utils/CMakeLists.txt:15 (find_package)
adam-dziedzic commented 4 years ago

I built the boost library manually:

sudo apt install python3-dev
# Get boost to see the local /usr/bin/python3 with python3-dev
./bootstrap.sh --with-python=/usr/bin/python3
sudo ./b2 install

After that:

make -j ext_aby
ENCRYPTO_utils was not found: add ENCRYPTO_utils subdirectory
-- Found Boost: /home/dockuser/code/he-transformer/build-new/boost/src/ext_boost 
(found suitable version "1.69.0", minimum required is "1.66.0") 
found components: system thread chrono date_time atomic
RELIC was not found: add RELIC subdirectory
adam-dziedzic commented 4 years ago

More outputs from make install for he-transformer:

[  9%] Performing download step (download, verify and extract) for 'ext_boost'
-- verifying file...
       file='/home/ady/code/he-transformer-adam/build/boost/src/boost_1_69_0.tar.gz'
-- File already exists and hash match (skip download):
  file='/home/ady/code/he-transformer-adam/build/boost/src/boost_1_69_0.tar.gz'
  SHA256='9a2c2819310839ea373f42d69e733c339b4e9a19deab6bfec448281554aa4dbb'
-- extracting...
     src='/home/ady/code/he-transformer-adam/build/boost/src/boost_1_69_0.tar.gz'
     dst='/home/ady/code/he-transformer-adam/build/boost/src/ext_boost'
-- extracting... [tar xfz]
-- extracting... [analysis]
-- extracting... [rename]
-- extracting... [clean up]
-- extracting... done
[ 10%] No update step for 'ext_boost'
[ 10%] No patch step for 'ext_boost'
[ 11%] No configure step for 'ext_boost'
[ 11%] No build step for 'ext_boost'
[ 11%] No install step for 'ext_boost'
[ 12%] Completed 'ext_boost'
[ 13%] Built target ext_boost
Scanning dependencies of target ext_aby
[ 13%] Creating directories for 'ext_aby'
[ 14%] Performing download step (git clone) for 'ext_aby'
Cloning into 'ext_aby'...
Already on 'public'
Your branch is up to date with 'origin/public'.
Submodule 'extern/ENCRYPTO_utils' (https://github.com/encryptogroup/ENCRYPTO_utils.git) registered for path 'extern/ENCRYPTO_utils'
Submodule 'extern/OTExtension' (https://github.com/encryptogroup/OTExtension.git) registered for path 'extern/OTExtension'
Cloning into '/mnt/c/Users/adzie/code/he-transformer-adam/build/ext_aby/src/ext_aby/extern/ENCRYPTO_utils'...
Cloning into '/mnt/c/Users/adzie/code/he-transformer-adam/build/ext_aby/src/ext_aby/extern/OTExtension'...
Submodule path 'extern/ENCRYPTO_utils': checked out 'a596166d9ebdc506d1ecb2394fac5ee5d598297b'
Submodule 'extern/googletest' (https://github.com/google/googletest.git) registered for path 'extern/ENCRYPTO_utils/extern/googletest'
Submodule 'extern/relic' (https://github.com/relic-toolkit/relic.git) registered for path 'extern/ENCRYPTO_utils/extern/relic'
Cloning into '/mnt/c/Users/adzie/code/he-transformer-adam/build/ext_aby/src/ext_aby/extern/ENCRYPTO_utils/extern/googletest'...
Cloning into '/mnt/c/Users/adzie/code/he-transformer-adam/build/ext_aby/src/ext_aby/extern/ENCRYPTO_utils/extern/relic'...
Submodule path 'extern/ENCRYPTO_utils/extern/googletest': checked out 'd1c1aac78160ae31353d9fe1bb1171353986a4f1'
Submodule path 'extern/ENCRYPTO_utils/extern/relic': checked out 'c25edf6520e5e4b39663f0d0594db7186440be5c'
Submodule path 'extern/OTExtension': checked out '2057fb0d4cfd39674e7813bc98d6d0a863138fc4'
Submodule 'extern/ENCRYPTO_utils' (https://github.com/encryptogroup/ENCRYPTO_utils.git) registered for path 'extern/OTExtension/extern/ENCRYPTO_utils'
Cloning into '/mnt/c/Users/adzie/code/he-transformer-adam/build/ext_aby/src/ext_aby/extern/OTExtension/extern/ENCRYPTO_utils'...
Submodule path 'extern/OTExtension/extern/ENCRYPTO_utils': checked out 'a596166d9ebdc506d1ecb2394fac5ee5d598297b'
Submodule 'extern/googletest' (https://github.com/google/googletest.git) registered for path 'extern/OTExtension/extern/ENCRYPTO_utils/extern/googletest'
Submodule 'extern/relic' (https://github.com/relic-toolkit/relic.git) registered for path 'extern/OTExtension/extern/ENCRYPTO_utils/extern/relic'
Cloning into '/mnt/c/Users/adzie/code/he-transformer-adam/build/ext_aby/src/ext_aby/extern/OTExtension/extern/ENCRYPTO_utils/extern/googletest'...
Cloning into '/mnt/c/Users/adzie/code/he-transformer-adam/build/ext_aby/src/ext_aby/extern/OTExtension/extern/ENCRYPTO_utils/extern/relic'...
Submodule path 'extern/OTExtension/extern/ENCRYPTO_utils/extern/googletest': checked out 'd1c1aac78160ae31353d9fe1bb1171353986a4f1'
Submodule path 'extern/OTExtension/extern/ENCRYPTO_utils/extern/relic': checked out 'c25edf6520e5e4b39663f0d0594db7186440be5c'
[ 14%] No update step for 'ext_aby'
[ 15%] No patch step for 'ext_aby'
[ 16%] Performing configure step for 'ext_aby'
-- The CXX compiler identification is GNU 8.4.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/g++-8 - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
ENCRYPTO_utils was not found: add ENCRYPTO_utils subdirectory
-- Found GMP: /usr/lib/x86_64-linux-gnu/libgmp.so
-- Found GMPXX: /usr/lib/x86_64-linux-gnu/libgmpxx.so
-- Found OpenSSL: /usr/lib/x86_64-linux-gnu/libcrypto.so (found version "1.1.1")
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
CMake Error at /usr/local/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
  Could NOT find Boost (missing: system thread) (found suitable version
  "1.69.0", minimum required is "1.66.0")
Call Stack (most recent call first):
  /usr/local/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:458 (_FPHSA_FAILURE_MESSAGE)
  /usr/local/share/cmake-3.18/Modules/FindBoost.cmake:2177 (find_package_handle_standard_args)
  extern/ENCRYPTO_utils/CMakeLists.txt:15 (find_package)

-- Configuring incomplete, errors occurred!
See also "/home/ady/code/he-transformer-adam/build/ext_aby/src/ext_aby-build/CMakeFiles/CMakeOutput.log".
CMakeFiles/ext_aby.dir/build.make:125: recipe for target 'ext_aby/src/ext_aby-stamp/ext_aby-configure' failed
make[2]: *** [ext_aby/src/ext_aby-stamp/ext_aby-configure] Error 1
CMakeFiles/Makefile2:487: recipe for target 'CMakeFiles/ext_aby.dir/all' failed
make[1]: *** [CMakeFiles/ext_aby.dir/all] Error 2
Makefile:148: recipe for target 'all' failed
make: *** [all] Error 2
adam-dziedzic commented 4 years ago

After building boost manually:

~/code/he-transformer/build$ **make ext_aby**
[ 50%] Built target ext_boost
[ 62%] Performing configure step for 'ext_aby'
ENCRYPTO_utils was not found: add ENCRYPTO_utils subdirectory
-- Found Boost: /home/ady/code/he-transformer-adam/build/boost/src/ext_boost (found suitable version "1.69.0", minimum required is "1.66.0") found components: system thread chrono date_time atomic
RELIC was not found: add RELIC subdirectory
CMake Warning (dev) at extern/ENCRYPTO_utils/CMakeLists.txt:40 (set):
  implicitly converting 'INTEGER' to 'STRING' type.
This warning is for project developers.  Use -Wno-dev to suppress it.
adam-dziedzic commented 3 years ago

For the latest version, I got the following error: image

adam-dziedzic commented 3 years ago

ERROR:/home/user/.cache/bazel/_bazel_nicolas/4baf5eb2dd7ac89f621f051ae39e488c/external/grpc/BUILD:507:1:C++ compilation of rule '@grpc//:gpr_base' failed (Exit 1)external/grpc/src/core/lib/gpr/log_linux.cc:43:13:error: ambiguating new declaration of 'long int gettid()' static long gettid(void) { returnsyscall(NR_gettid); }             ^~In file includedfrom /usr/include/unistd.h:1170,                 fromexternal/grpc/src/core/lib/gpr/log_linux.cc:41:/usr/include/x86_64-linux-gnu/bits/unistd_ext.h:34:16:note: old declaration 'pid_t gettid()' extern pid_t gettid (void) __THROW;                ^~external/grpc/src/core/lib/gpr/log_linux.cc:43:13:warning: 'long int gettid()' defined but not used [-Wunused-function] static long gettid(void) { returnsyscall(NR_gettid); }             ^~Target//tensorflow/tools/pip_package:build_pip_package failed to buildUse--verbose_failures to see the command lines of failed build steps.