Building from Docker on clean machine fails

[cchoquette]

In trying to build this project using the docker images, with the following commands:

make check_gss OS_ID=ubuntu1804 RM_CONTAINER=false and make check_gss OS_ID=ubuntu1804

on a clean matching with ubuntu 20 OS and docker version 19.03.12

I run into the error pasted below. I tried using the details in #45 to resolve this issue but have not been able to install the docker image to our clean machine.

`Installing collected packages: ngraph-tensorflow-bridge Successfully installed ngraph-tensorflow-bridge-0.22.0rc3 /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/venv-tf-py3/lib/ python3.6/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1 type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, ( 1,)) / '(1,)type'. _np_qint8 = np.dtype([("qint8", np.int8, 1)]) /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/venv-tf-py3/lib/ python3.6/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1 type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, ( 1,)) / '(1,)type'. _np_quint8 = np.dtype([("quint8", np.uint8, 1)]) /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/venv-tf-py3/lib/ python3.6/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1 type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, ( 1,)) / '(1,)type'. _np_qint16 = np.dtype([("qint16", np.int16, 1)]) /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/venv-tf-py3/lib/ python3.6/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1 type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, ( 1,)) / '(1,)type'. _np_quint16 = np.dtype([("quint16", np.uint16, 1)]) /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/venv-tf-py3/lib/ python3.6/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1 type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, ( 1,)) / '(1,)type'. _np_qint32 = np.dtype([("qint32", np.int32, 1)]) /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/venv-tf-py3/lib/ python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1 type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, ( 1,)) / '(1,)type'. np_resource = np.dtype([("resource", np.ubyte, 1)]) /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/venv-tf-py3/lib/ python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1 ) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as ( type, (1,)) / '(1,)type'. _np_qint8 = np.dtype([("qint8", np.int8, 1)]) /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/venv-tf-py3/lib/ python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1 ) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as ( type, (1,)) / '(1,)type'. _np_quint8 = np.dtype([("quint8", np.uint8, 1)]) /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/venv-tf-py3/lib/ python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1 ) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as ( type, (1,)) / '(1,)type'. _np_qint16 = np.dtype([("qint16", np.int16, 1)]) /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/venv-tf-py3/lib/ python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1 ) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as ( type, (1,)) / '(1,)type'. _np_quint16 = np.dtype([("quint16", np.uint16, 1)]) /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/venv-tf-py3/lib/ python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1 ) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as ( type, (1,)) / '(1,)type'. _np_qint32 = np.dtype([("qint32", np.int32, 1)]) /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/venv-tf-py3/lib/ python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1 ) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as ( type, (1,)) / '(1,)type'. np_resource = np.dtype([("resource", np.ubyte, 1)]) ARTIFACTS location: /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/build_cm ake/artifacts Loading virtual environment from: /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngra ph_tf/build_cmake/venv-tf-py3 Loading virtual environment from: /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngra ph_tf/build_cmake/venv-tf-py3 PIP location Target Arch: native Building TensorFlow from source PYTHON_BIN_PATH: /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake /venv-tf-py3/bin/python SOURCE DIR: /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/tens orflow ARTIFACTS DIR: /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/a rtifacts/tensorflow TF Wheel: /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/artifa cts/tensorflow/tensorflow-1.14.0-cp36-cp36m-linux_x86_64.whl PYTHON_BIN_PATH: /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake /venv-tf-py3/bin/python SOURCE DIR: /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/tens orflow ARTIFACTS DIR: /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/a rtifacts/tensorflow Cannot remove: /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/a rtifacts/tensorflow/libtensorflow_cc.so.1 Copying bazel-bin/tensorflow/libtensorflow_cc.so.1 to /home/dockuser/he-transformer-test/BUILD-GCC/ext_ng raph_tf/src/ext_ngraph_tf/build_cmake/artifacts/tensorflow Loading virtual environment from: /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngra ph_tf/build_cmake/venv-tf-py3 LIB: /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/venv-tf-py3 /lib/python3.6/site-packages/tensorflow CXX_ABI: 1 Using local nGraph source in directory /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph/src/ext_n graph Source location: /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph/src/ext_ngraph Running COMMAND: cmake -DNGRAPH_INSTALL_PREFIX=/home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf /src/ext_ngraph_tf/build_cmake/artifacts -DNGRAPH_USE_CXX_ABI=1 -DNGRAPH_DEX_ONLY=TRUE -DNGRAPH_DEBUG_ENA BLE=NO -DNGRAPH_UNIT_TEST_ENABLE=NO -DNGRAPH_TARGET_ARCH=native -DNGRAPH_TUNE_ARCH=native -DNGRAPH_TBB_EN ABLE=FALSE -DNGRAPH_DISTRIBUTED_ENABLE=OFF -DNGRAPH_TOOLS_ENABLE=YES -DNGRAPH_GPU_ENABLE=NO -DNGRAPH_PLAI DML_ENABLE=NO -DNGRAPH_INTELGPU_ENABLE=NO /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph/src/ext _ngraph Running COMMAND: make -j20 Running COMMAND: make install TF_SRC_DIR: /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/ten sorflow Loading virtual environment from: /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngra ph_tf/build_cmake/venv-tf-py3 Source location: /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf OUTPUT WHL FILE: ngraph_tensorflow_bridge-0.22.0rc3-py2.py3-none-manylinux1_x86_64.whl OUTPUT WHL DST: /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/ artifacts/ngraph_tensorflow_bridge-0.22.0rc3-py2.py3-none-manylinux1_x86_64.whl SUCCESSFULLY generated wheel: ngraph_tensorflow_bridge-0.22.0rc3-py2.py3-none-manylinux1_x86_64.whl PWD: /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake Running COMMAND: cp -r /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/build _cmake/tensorflow/tensorflow/python /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ng raph_tf/build_cmake/artifacts/tensorflow Loading virtual environment from: /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngra ph_tf/build_cmake/venv-tf-py3 Version information TensorFlow version: 1.14.0 C Compiler version used in building TensorFlow: 7.5.0 nGraph bridge version: b'0.22.0-rc3' nGraph version used for this build: b'0.28.0-rc.1+d2cd873' TensorFlow version used for this build: v1.14.0-0-g87989f6959 CXX11_ABI flag used for this build: 1 nGraph bridge built with Grappler: True nGraph bridge built with Variables and Optimizers Enablement: False nGraph bridge built with Distributed Build: 0 Build successful cd /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf && /usr/local/lib/python3 .6/dist-packages/cmake/data/bin/cmake -E touch /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf /src/ext_ngraph_tf-stamp/ext_ngraph_tf-build [100%] Performing install step for 'ext_ngraph_tf' cd /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf && ln -fs /home/dockuser/ he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/venv-tf-py3 /home/dockuser/he-t ransformer-test/BUILD-GCC/external cd /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf && /usr/local/lib/python3 .6/dist-packages/cmake/data/bin/cmake -E touch /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf /src/ext_ngraph_tf-stamp/ext_ngraph_tf-install [100%] Completed 'ext_ngraph_tf' /usr/local/lib/python3.6/dist-packages/cmake/data/bin/cmake -E make_directory /home/dockuser/he-transform er-test/BUILD-GCC/CMakeFiles /usr/local/lib/python3.6/dist-packages/cmake/data/bin/cmake -E touch /home/dockuser/he-transformer-test/B UILD-GCC/CMakeFiles/ext_ngraph_tf-complete /usr/local/lib/python3.6/dist-packages/cmake/data/bin/cmake -E touch /home/dockuser/he-transformer-test/B UILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf-stamp/ext_ngraph_tf-done make[2]: Leaving directory '/home/dockuser/he-transformer-test/BUILD-GCC' [100%] Built target ext_ngraph_tf make[1]: Leaving directory '/home/dockuser/he-transformer-test/BUILD-GCC' /usr/local/lib/python3.6/dist-packages/cmake/data/bin/cmake -E cmake_progress_start /home/dockuser/he-tra nsformer-test/BUILD-GCC/CMakeFiles 0 make -f CMakeFiles/Makefile2 preinstall make[1]: Entering directory '/home/dockuser/he-transformer-test/BUILD-GCC' make[1]: Nothing to be done for 'preinstall'. make[1]: Leaving directory '/home/dockuser/he-transformer-test/BUILD-GCC' Install the project... /usr/local/lib/python3.6/dist-packages/cmake/data/bin/cmake -P cmake_install.cmake -- Install configuration: "RelWithDebInfo" -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/libdnnl.so -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/libngraph_test_util.a -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/libngraph.so -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/libmklml_intel.so -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/libcpu_backend.so -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/libiomp5.so -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/libngraph_bridge.so -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/libnop_backend.so -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/libinterpreter_backend.so -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/libngraph_bridge_device.so -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/libgcpu_backend.so -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/include/ngraph_backend_manager.h -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/include/ngraph_log.h -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/include/ngraph/version.hpp -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/libz.so.1.2.11 -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/libz.so.1 -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/libz.so -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/CMakeFiles -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/CMakeFiles/zlibstatic.dir -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/CMakeFiles/minigzip.dir -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/CMakeFiles/minigzip.dir/test -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/CMakeFiles/CheckTypeSize -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/CMakeFiles/example.dir -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/CMakeFiles/example.dir/test -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/CMakeFiles/minigzip64.dir -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/CMakeFiles/minigzip64.dir/test -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/CMakeFiles/3.18.2 -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/CMakeFiles/3.18.2/CompilerIdC -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/CMakeFiles/3.18.2/CompilerIdC/tm p -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/CMakeFiles/CMakeTmp -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/CMakeFiles/zlib.dir -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/CMakeFiles/example64.dir -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/CMakeFiles/example64.dir/test -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/libz.a -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/external/lib/libhe_seal_backend.so -- Set runtime path of "/home/dockuser/he-transformer-test/BUILD-GCC/external/lib/libhe_seal_backend.so" to "$ORIGIN" -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/v env-tf-py3/lib/python3.6/site-packages/ngraph_bridge/libhe_seal_backend.so -- Set runtime path of "/home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/buil d_cmake/venv-tf-py3/lib/python3.6/site-packages/ngraph_bridge/libhe_seal_backend.so" to "$ORIGIN" -- Installing: /home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/a rtifacts/lib/libhe_seal_backend.so -- Set runtime path of "/home/dockuser/he-transformer-test/BUILD-GCC/ext_ngraph_tf/src/ext_ngraph_tf/buil d_cmake/artifacts/lib/libhe_seal_backend.so" to "$ORIGIN" -- Installing: /usr/local/lib/libhe_seal_backend.so CMake Error at src/cmake_install.cmake:153 (file): file INSTALL cannot copy file "/home/dockuser/he-transformer-test/BUILD-GCC/src/libhe_seal_backend.so" to "/usr/local/lib/libhe_seal_backend.so": Success. Call Stack (most recent call first): cmake_install.cmake:83 (include)

Makefile:104: recipe for target 'install' failed make: [install] Error 1 make: [Makefile:157: build_gcc] Error 2 $USER@equus:\~/Documents/he-transformer/contrib/docker$ ls build_docker_image.sh Dockerfile.he_transformer.fedora28 Makefile build-he-transformer-and-test.sh Dockerfile.he_transformer.ubuntu1604 ngraph-tf.cmake CMakeLists.txt Dockerfile.he_transformer.ubuntu1804 README.md docker_cleanup.sh fix_numpy_for_tf.patch run_as_centos_user.sh Dockerfile make-dimage.sh run_as_fedora_user.sh Dockerfile.he_transformer.centos74 make_docker_image.sh run_as_ubuntu_user.sh $USER@equus:\~/Documents/he-transformer/contrib/docker$`

[adam-dziedzic]

@fboemer Hi Fabian, could you please look into this issue with copying the file. The dockuser is given sudo access (in contrib/docker/run_as_ubuntu_user.sh) but this command: file INSTALL cannot copy file "/home/dockuser/he-transformer-test/BUILD-GCC/src/libhe_seal_backend.so" to "/usr/local/lib/libhe_seal_backend.so" is run without sudo and it fails.

[adam-dziedzic]

This might solve the problem with sudo access: https://github.com/IntelAI/he-transformer/blob/be835adf0aa0953f0eafab734080add58249b241/contrib/docker/run_as_ubuntu_user.sh#L88

[PereteanuGeorge]

@adam-dziedzic @fboemer Hello, I am getting the same error as above while trying to install he-transformer with docker. I have uncommented the line https://github.com/IntelAI/he-transformer/blob/be835adf0aa0953f0eafab734080add58249b241/contrib/docker/run_as_ubuntu_user.sh#L90 and moreover added "RUN useradd -m docker && echo "docker:docker" | chpasswd && adduser docker sudo" in the dockerfile. Unfortunately, I still get the same error about permission. I was wondering if there is any solution to this issue. Thank you!

[adam-dziedzic]

Check my answer here: https://github.com/IntelAI/he-transformer/issues/51

[PereteanuGeorge]

@adam-dziedzic Thank you so much for your fast reply!

So I have installed ABY and I already had g++ version 9.5 and installed boost library. After installing all the libraries I have mentioned, I ran again "make check_all OS_ID=ubuntu1804", should this work? Thanks

however, I didn't really get the point of how to install he-transformer using ABY. Do I just run again the command to make check_all? Thanks a lot for your help!

[adam-dziedzic]

After that, I installed all the targets in the makefile separately until I could do make install python_client.

[PereteanuGeorge]

@adam-dziedzic do you mind contacting me at gp3917@ic.ac.uk so that I don't spam here? I was wondering if you have a VM with he-transformer already installed where I could test my code since literally I cant install it. Thanks

[hnamkung15]

Hi, I experience the same permission denied problem. Is this issue resolved?