intel / he-transformer

nGraph-HE: Deep learning with Homomorphic Encryption (HE) through Intel nGraph
Apache License 2.0
172 stars 35 forks source link

Install/Build failure -- tensorflow rule compilation error #36

Closed mlayou closed 4 years ago

mlayou commented 4 years ago

Hi,

When running make install I get the error below. Any idea what's causing it and how to fix it?

Thanks.

ERROR: /home/user1/nGraph/he-transformer/build/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/tensorflow/tensorflow/core/kernels/BUILD:3255:1: C++ compilation of rule '//tensorflow/core/kernels:matrix_square_root_op' failed (Exit 4)
gcc: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.
Target //tensorflow/tools/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 4483.868s, Critical Path: 238.51s
INFO: 3757 processes: 3757 local.
FAILED: Build did NOT complete successfully
Traceback (most recent call last):
  File "/home/user1/nGraph/he-transformer/build/ext_ngraph_tf/src/ext_ngraph_tf/build_ngtf.py", line 525, in <module>
    main()
  File "/home/user1/nGraph/he-transformer/build/ext_ngraph_tf/src/ext_ngraph_tf/build_ngtf.py", line 328, in main
    target_arch, verbosity)
  File "/home/user1/nGraph/he-transformer/build/ext_ngraph_tf/src/ext_ngraph_tf/tools/build_utils.py", line 232, in build_tensorflow
    command_executor(cmd)
  File "/home/user1/nGraph/he-transformer/build/ext_ngraph_tf/src/ext_ngraph_tf/tools/build_utils.py", line 60, in command_executor
    raise Exception("Error running command: " + cmd)
Exception: Error running command: bazel build --config=opt --config=noaws --config=nohdfs --config=noignite --config=nokafka --config=nonccl //tensorflow/tools/pip_package:build_pip_package
CMakeFiles/ext_ngraph_tf.dir/build.make:112: recipe for target 'ext_ngraph_tf/src/ext_ngraph_tf-stamp/ext_ngraph_tf-build' failed
make[2]: *** [ext_ngraph_tf/src/ext_ngraph_tf-stamp/ext_ngraph_tf-build] Error 1
CMakeFiles/Makefile2:464: recipe for target 'CMakeFiles/ext_ngraph_tf.dir/all' failed
make[1]: *** [CMakeFiles/ext_ngraph_tf.dir/all] Error 2
Makefile:129: recipe for target 'all' failed
make: *** [all] Error 2
mlayou commented 4 years ago

The problem above is caused by the fact that my VM run out of RAM. I've added more RAM to the VM and the problem was fixed.

acoj1993 commented 4 years ago

Hi, I am experiencing exactly the same issue. Can you tell me, how much RAM was sufficient in your case? I have been monitoring the RAM during the installation and for me it didn't seem as if it was running out of memory. I allocated 12 GB of RAM to the VM.

Maybe I am wrong about getting the same error. Here is the output I get:

tensorflow/python/lib/core/bfloat16.cc:608:60: note:   no known conversion for argument 2 from '<unresolved overloaded function type>' to 'PyUFuncGenericFunction {aka void (*)(char**, const long int*, const long int*, void*)}'
Target //tensorflow/tools/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 5481.302s, Critical Path: 245.57s
INFO: 6071 processes: 6071 local.
FAILED: Build did NOT complete successfully
Traceback (most recent call last):
  File "/home/aleks/he-transformer/build/ext_ngraph_tf/src/ext_ngraph_tf/build_ngtf.py", line 525, in <module>
    main()
  File "/home/aleks/he-transformer/build/ext_ngraph_tf/src/ext_ngraph_tf/build_ngtf.py", line 328, in main
    target_arch, verbosity)
  File "/home/aleks/he-transformer/build/ext_ngraph_tf/src/ext_ngraph_tf/tools/build_utils.py", line 231, in build_tensorflow
    command_executor(cmd)
  File "/home/aleks/he-transformer/build/ext_ngraph_tf/src/ext_ngraph_tf/tools/build_utils.py", line 60, in command_executor
    raise Exception("Error running command: " + cmd)
Exception: Error running command: bazel build --config=opt --config=noaws --config=nohdfs --config=noignite --config=nokafka --config=nonccl //tensorflow/tools/pip_package:build_pip_package
CMakeFiles/ext_ngraph_tf.dir/build.make:111: recipe for target 'ext_ngraph_tf/src/ext_ngraph_tf-stamp/ext_ngraph_tf-build' failed
make[2]: *** [ext_ngraph_tf/src/ext_ngraph_tf-stamp/ext_ngraph_tf-build] Error 1
CMakeFiles/Makefile2:402: recipe for target 'CMakeFiles/ext_ngraph_tf.dir/all' failed
make[1]: *** [CMakeFiles/ext_ngraph_tf.dir/all] Error 2
Makefile:129: recipe for target 'all' failed
make: *** [all] Error 2
mlayou commented 4 years ago

Hi, In my case, 6GB of RAM were enough to do the job. The first line in the logs above suggests that this error might be different from the one I had. Maybe double check that you have all of the dependencies installed prior to starting the build.