search

alexgkendall / SegNet-Tutorial

Files for a tutorial to train SegNet for road scenes using the CamVid dataset
http://mi.eng.cam.ac.uk/projects/segnet/tutorial.html
847 stars 517 forks source link

Docker Build Fails #158

Closed KeunwooPark closed 5 years ago

KeunwooPark commented 5 years ago

I tried to build the gpu docker image on Ubuntu 16.04. Dockerfile fails with the below error.

[  1%] Built target proto
[  1%] [  2%] [  2%] [  2%] [  4%] [  5%] [  5%] [  5%] Building NVCC (Device) object src/caffe/CMakeFiles/cuda_compile.dir/util/./cuda_compile_generated_im2col.cu.o
Building NVCC (Device) object src/caffe/CMakeFiles/cuda_compile.dir/util/./cuda_compile_generated_math_functions.cu.o
Building NVCC (Device) object src/caffe/CMakeFiles/cuda_compile.dir/layers/./cuda_compile_generated_exp_layer.cu.o
Building NVCC (Device) object src/caffe/CMakeFiles/cuda_compile.dir/layers/./cuda_compile_generated_sigmoid_cross_entropy_loss_layer.cu.o
Building NVCC (Device) object src/caffe/CMakeFiles/cuda_compile.dir/layers/./cuda_compile_generated_slice_layer.cu.o
Building NVCC (Device) object src/caffe/CMakeFiles/cuda_compile.dir/layers/./cuda_compile_generated_cudnn_softmax_layer.cu.o
Building NVCC (Device) object src/caffe/CMakeFiles/cuda_compile.dir/layers/./cuda_compile_generated_lrn_layer.cu.o
Building NVCC (Device) object src/caffe/CMakeFiles/cuda_compile.dir/layers/./cuda_compile_generated_reduction_layer.cu.o
/opt/caffe/include/caffe/util/cudnn.hpp(123): error: argument of type "int" is incompatible with parameter of type "cudnnNanPropagation_t"

/opt/caffe/include/caffe/util/cudnn.hpp(123): error: too few arguments in function call

2 errors detected in the compilation of "/tmp/tmpxft_00000502_00000000-14_im2col.cpp4.ii".
CMake Error at cuda_compile_generated_im2col.cu.o.cmake:264 (message):
  Error generating file
  /opt/caffe/build/src/caffe/CMakeFiles/cuda_compile.dir/util/./cuda_compile_generated_im2col.cu.o

make[2]: *** [src/caffe/CMakeFiles/cuda_compile.dir/util/./cuda_compile_generated_im2col.cu.o] Error 1
make[2]: *** Waiting for unfinished jobs....
/opt/caffe/include/caffe/util/cudnn.hpp(123): error: argument of type "int" is incompatible with parameter of type "cudnnNanPropagation_t"

/opt/caffe/include/caffe/util/cudnn.hpp(123): error: too few arguments in function call

/opt/caffe/include/caffe/util/cudnn.hpp(123): error: argument of type "int" is incompatible with parameter of type "cudnnNanPropagation_t"

/opt/caffe/include/caffe/util/cudnn.hpp(123): error: too few arguments in function call

/opt/caffe/include/caffe/util/cudnn.hpp(123): error: argument of type "int" is incompatible with parameter of type "cudnnNanPropagation_t"

/opt/caffe/include/caffe/util/cudnn.hpp(123): error: too few arguments in function call

/opt/caffe/include/caffe/util/cudnn.hpp(123): error: argument of type "int" is incompatible with parameter of type "cudnnNanPropagation_t"

/opt/caffe/include/caffe/util/cudnn.hpp(123): error: too few arguments in function call

2 errors detected in the compilation of "/tmp/tmpxft_0000052a_00000000-14_lrn_layer.cpp4.ii".
/opt/caffe/include/caffe/util/cudnn.hpp(123): error: argument of type "int" is incompatible with parameter of type "cudnnNanPropagation_t"

/opt/caffe/include/caffe/util/cudnn.hpp(123): error: too few arguments in function call

CMake Error at cuda_compile_generated_lrn_layer.cu.o.cmake:264 (message):
  Error generating file
  /opt/caffe/build/src/caffe/CMakeFiles/cuda_compile.dir/layers/./cuda_compile_generated_lrn_layer.cu.o

make[2]: *** [src/caffe/CMakeFiles/cuda_compile.dir/layers/./cuda_compile_generated_lrn_layer.cu.o] Error 1
2 errors detected in the compilation of "/tmp/tmpxft_00000532_00000000-14_exp_layer.cpp4.ii".
CMake Error at cuda_compile_generated_exp_layer.cu.o.cmake:264 (message):
  Error generating file
  /opt/caffe/build/src/caffe/CMakeFiles/cuda_compile.dir/layers/./cuda_compile_generated_exp_layer.cu.o

make[2]: *** [src/caffe/CMakeFiles/cuda_compile.dir/layers/./cuda_compile_generated_exp_layer.cu.o] Error 1
2 errors detected in the compilation of "/tmp/tmpxft_0000051b_00000000-14_slice_layer.cpp4.ii".
CMake Error at cuda_compile_generated_slice_layer.cu.o.cmake:264 (message):
  Error generating file
  /opt/caffe/build/src/caffe/CMakeFiles/cuda_compile.dir/layers/./cuda_compile_generated_slice_layer.cu.o

make[2]: *** [src/caffe/CMakeFiles/cuda_compile.dir/layers/./cuda_compile_generated_slice_layer.cu.o] Error 1
2 errors detected in the compilation of "/tmp/tmpxft_00000510_00000000-14_sigmoid_cross_entropy_loss_layer.cpp4.ii".
CMake Error at cuda_compile_generated_sigmoid_cross_entropy_loss_layer.cu.o.cmake:264 (message):
  Error generating file
  /opt/caffe/build/src/caffe/CMakeFiles/cuda_compile.dir/layers/./cuda_compile_generated_sigmoid_cross_entropy_loss_layer.cu.o

make[2]: *** [src/caffe/CMakeFiles/cuda_compile.dir/layers/./cuda_compile_generated_sigmoid_cross_entropy_loss_layer.cu.o] Error 1
2 errors detected in the compilation of "/tmp/tmpxft_0000051e_00000000-14_reduction_layer.cpp4.ii".
CMake Error at cuda_compile_generated_reduction_layer.cu.o.cmake:264 (message):
  Error generating file
  /opt/caffe/build/src/caffe/CMakeFiles/cuda_compile.dir/layers/./cuda_compile_generated_reduction_layer.cu.o

make[2]: *** [src/caffe/CMakeFiles/cuda_compile.dir/layers/./cuda_compile_generated_reduction_layer.cu.o] Error 1
/opt/caffe/include/caffe/util/cudnn.hpp(123): error: argument of type "int" is incompatible with parameter of type "cudnnNanPropagation_t"

/opt/caffe/include/caffe/util/cudnn.hpp(123): error: too few arguments in function call

2 errors detected in the compilation of "/tmp/tmpxft_00000550_00000000-14_cudnn_softmax_layer.cpp4.ii".
CMake Error at cuda_compile_generated_cudnn_softmax_layer.cu.o.cmake:264 (message):
  Error generating file
  /opt/caffe/build/src/caffe/CMakeFiles/cuda_compile.dir/layers/./cuda_compile_generated_cudnn_softmax_layer.cu.o

make[2]: *** [src/caffe/CMakeFiles/cuda_compile.dir/layers/./cuda_compile_generated_cudnn_softmax_layer.cu.o] Error 1
/opt/caffe/include/caffe/util/cudnn.hpp(123): error: argument of type "int" is incompatible with parameter of type "cudnnNanPropagation_t"

/opt/caffe/include/caffe/util/cudnn.hpp(123): error: too few arguments in function call

2 errors detected in the compilation of "/tmp/tmpxft_00000544_00000000-14_math_functions.cpp4.ii".
CMake Error at cuda_compile_generated_math_functions.cu.o.cmake:264 (message):
  Error generating file
  /opt/caffe/build/src/caffe/CMakeFiles/cuda_compile.dir/util/./cuda_compile_generated_math_functions.cu.o

make[2]: *** [src/caffe/CMakeFiles/cuda_compile.dir/util/./cuda_compile_generated_math_functions.cu.o] Error 1
make[1]: *** [src/caffe/CMakeFiles/caffe.dir/all] Error 2
make: *** [all] Error 2
The command '/bin/sh -c git clone -b ${CLONE_TAG} --depth 1 https://github.com/alexgkendall/caffe-segnet.git . &&     for req in $(cat python/requirements.txt) pydot; do pip install $req; done &&     mkdir build && cd build &&     cmake -DUSE_CUDNN=1 .. &&     make -j"$(nproc)"' returned a non-zero code: 2
KeunwooPark commented 5 years ago

I found out what was wrong. I was using GTX 1060 and it had Pascal Architecture. So I had to use cudnn 5, not 4. Even though the Dockerfile for gpu uses cudnn5, it does not use Caffe SegNet repo for cudnn 5. The solution is to change the git repo in the Docker file to caffe-segnet-cudnn5. Moreover, I had to use cuda 8.0 and Ubuntu 16.04 for the Docker environment. I think it is because of the GPU architecture issue.