tensorflow / models

Models and examples built with TensorFlow
Other
77.25k stars 45.75k forks source link

Tensorflow Bazel Build Failure - GCC failing #1261

Closed alfydavy closed 6 years ago

alfydavy commented 7 years ago

/tensorflow/tensorflow/core/kernels/BUILD:2122:1 C++ compilation of rule '//tensorflow/core/kernels:argmax_op' failed: gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -B/usr/bin -B/usr/bin -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG ... (remaining 130 argument(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 4. gcc: internal compiler error: Killed (program cc1plus)

alfydavy commented 7 years ago

My System is Mac OS running sierra 10.12.3

aselle commented 7 years ago

are you possibly running out of memory? how much memory does your system have?

alfydavy commented 7 years ago

System has 16 gigs of memory there was no out of memory exception just the gcc failure alert

Sent from my iPhone

On 29-Mar-2017, at 02:44, Andrew Selle notifications@github.com wrote:

are you possibly running out of memory? how much memory does your system have?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

aselle commented 7 years ago

that's strange. Could you try compiling a simple c++ program and make sure your compiler is working

cat <<EOF > foo.cpp
#include <iostream>
int main(){
  std::cout<<"My compiler works."<<std::endl;
  return 0;
}
EOF
/usr/bin/gcc foo.cpp

After that what bazel version? Try upgrading to the latest? What tensorflow version? What clang version (or is it really gcc)?

Thanks!

ledovsky commented 7 years ago

I have a problem that looks similar

ERROR: /root/.cache/bazel/_bazel_root/821bda42e7dedbfae9d936785dd2d125/external/org_tensorflow/tensorflow/core/kernels/BUILD:1941:1: C++ compilation of rule '@org_tensorflow//tensorflow/core/kernels:qr_op' failed: gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -Wall -Wl,-z,-relro,-z,now -B/usr/bin -B/usr/bin -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-canonical-system-headers ... (remaining 112 argument(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 4. gcc: internal compiler error: Killed (program cc1plus)

I build via Docker with standard Dockerfile and have MacBook Pro with 8Gb RAM, OS X Sierra 10.12.4

bogatyy commented 7 years ago

@ledovsky Not sure where the error comes from, but does not seem DRAGNN-specific. Can you build other stuff successfully, e.g. pure TensorFlow?

Also, since you're using Docker, you don't have to re-build anything, all the binaries are pre-compiled in the image (just make sure to pull the latest image). Try just running something out of ./bazel-bin/...

modanesh commented 7 years ago

@aselle I have the same problem as @alfydavy. My bazel version is:

Build label: 0.4.5-homebrew My clang version is:

Apple LLVM version 8.1.0 (clang-802.0.38) I face the following error:

ERROR: /tensorflow/tensorflow/contrib/verbs/BUILD:135:1: C++ compilation of rule '//tensorflow/contrib/verbs:rdma' failed: cc_wrapper.sh failed: error executing command external/local_config_cc/cc_wrapper.sh -U_FORTIFY_SOURCE -fstack-protector -Wall -Wthread-safety -Wself-assign -fcolor-diagnostics -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG ... (remaining 139 argument(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.

barrylachapelle commented 7 years ago

+1 ERROR: /tensorflow/tensorflow/core/kernels/BUILD:2297:1: C++ compilation of rule '//tensorflow/core/kernels:self_adjoint_eig_v2_op' failed: gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -B/usr/bin -B/usr/bin -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG ... (remaining 124 argument(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 4. gcc: internal compiler error: Killed (program cc1plus)

karmel commented 6 years ago

Automatically closing due to lack of recent activity. Since this issue is old at this point, please reopen the issue if it still occurs when tried with the latest version of Tensorflow. Thank you.

jason-tilley commented 6 years ago

I am also having this error on Ubuntu 16.04. My computer only has 4 GB of RAM, but 8 GB of VRAM:

ERROR: /tensorflow/tensorflow/core/kernels/BUILD:2882:1: C++ compilation of rule '//tensorflow/core/kernels:argmax_op' failed (Exit 4) gcc: internal compiler error: Killed (program cc1plus) Please submit a full bug report, with preprocessed source if appropriate. See <file:///usr/share/doc/gcc-5/README.Bugs> for instructions.

sourman commented 5 years ago

tldr: Build succeeded when I increased memory available to windows docker engine from 2048 MB to 4096 MB

I ran into this problem with the following configs Running on a windows machine with docker engine settings set to 2048 MB of RAM tensorflow version docker container tensorflow/tensorflow:1.12.0-devel

I installed in this container bazel 0.19.2 using the following dockerfile commands

RUN curl -L -O https://github.com/bazelbuild/bazel/releases/download/0.19.2/bazel-0.19.2-installer-linux-x86_64.sh
RUN chmod +x bazel-0.19.2-installer-linux-x86_64.sh
RUN ["./bazel-0.19.2-installer-linux-x86_64.sh"]

I then attempted to build the transform-graph tool using the following bazel build tensorflow/tools/graph_transforms:transform_graph​

Towards the end of the build I saw the compile failure mentioned in this issue. The only thing I did to fix it was to increase the memory available for the docker engine on windows from 2048 MB to 4096 MB