search

mathmanu / caffe-jacinto

This repository has moved. The new link can be obtained from https://github.com/TexasInstruments/jacinto-ai-devkit
116 stars 35 forks source link

Installation with Docker #3

Closed Wronskia closed 7 years ago

Wronskia commented 7 years ago

Hello,

I am trying to install your new caffe-0.16 version using Docker but I am running into few problems. Here is my Dockerfile :

+++++++++++++

FROM nvidia/cuda:8.0-cudnn6-devel-ubuntu16.04 LABEL maintainer caffe-maint@googlegroups.com

RUN apt-get update && apt-get install -y --no-install-recommends \ build-essential \ cmake \ git \ wget \ libatlas-base-dev \ libboost-all-dev \ libgflags-dev \ libgoogle-glog-dev \ libhdf5-serial-dev \ libleveldb-dev \ liblmdb-dev \ libopencv-dev \ libprotobuf-dev \ libsnappy-dev \ protobuf-compiler \ python-dev \ python-numpy \ python-pip \ python-setuptools \ python-scipy && \ rm -rf /var/lib/apt/lists/*

ENV CAFFE_ROOT=/opt/caffe-jacinto WORKDIR $CAFFE_ROOT

ENV CLONE_TAG=1.0

RUN git clone -b caffe-0.16 https://github.com/tidsp/caffe-jacinto

RUN git clone -b caffe-0.16 https://github.com/tidsp/caffe-jacinto-models

RUN cd caffe-jacinto && \ cp -f Makefile.config.example Makefile.config RUN cd caffe-jacinto && \ sed -i '/USE_NCCL/s/^#//g' Makefile.config RUN cd caffe-jacinto && \ sed -i '/WITH_PYTHON_LAYER/s/^#//g' Makefile.config RUN cd caffe-jacinto && \ sed -i '/USE_CUDNN/s/^#//g' Makefile.config

RUN pip install --upgrade pip && \ cd caffe-jacinto/python && for req in $(cat requirements.txt) pydot; do pip install $req; done && cd .. && \ git clone https://github.com/NVIDIA/nccl.git && cd nccl && make -j install && cd .. && rm -rf nccl && \ mkdir build && cd build && \ cmake -DUSE_CUDNN=1 -DUSE_NCCL=1 .. && \ make -j"$(nproc)"

ENV PYCAFFE_ROOT $CAFFE_ROOT/caffe-jacinto/python ENV PYTHONPATH $PYCAFFE_ROOT:$PYTHONPATH ENV PATH $CAFFE_ROOT/build/tools:$PYCAFFE_ROOT:$PATH RUN echo "$CAFFE_ROOT/build/lib" >> /etc/ld.so.conf.d/caffe.conf && ldconfig

WORKDIR /workspace

+++++++++++++++++

Here is the error I get ::

nvcc fatal : Unsupported gpu architecture 'compute_70' CMake Error at cuda_compile_generated_adagrad_solver.cu.o.cmake:203 (message): Error generating /opt/caffe-jacinto/caffe-jacinto/build/src/caffe/CMakeFiles/cuda_compile.dir/solvers/./cuda_compile_generated_adagrad_solver.cu.o

make[2]: [src/caffe/CMakeFiles/cuda_compile.dir/solvers/cuda_compile_generated_adagrad_solver.cu.o] Error 1 make[2]: Waiting for unfinished jobs.... src/caffe/CMakeFiles/caffe.dir/build.make:91: recipe for target 'src/caffe/CMakeFiles/cuda_compile.dir/solvers/cuda_compile_generated_adagrad_solver.cu.o' failed nvcc fatal : Unsupported gpu architecture 'compute_70' CMake Error at cuda_compile_generated_hdf5_data_layer.cu.o.cmake:203 (message): Error generating /opt/caffe-jacinto/caffe-jacinto/build/src/caffe/CMakeFiles/cuda_compile.dir/layers/./cuda_compile_generated_hdf5_data_layer.cu.o

nvcc fatal : Unsupported gpu architecture 'compute_70' nvcc fatal : Unsupported gpu architecture 'compute_70' CMake Error at cuda_compile_generated_nesterov_solver.cu.o.cmake:203 (message): Error generating /opt/caffe-jacinto/caffe-jacinto/build/src/caffe/CMakeFiles/cuda_compile.dir/solvers/./cuda_compile_generated_nesterov_solver.cu.o

src/caffe/CMakeFiles/caffe.dir/build.make:133: recipe for target 'src/caffe/CMakeFiles/cuda_compile.dir/layers/cuda_compile_generated_hdf5_data_layer.cu.o' failed make[2]: *** [src/caffe/CMakeFiles/cuda_compile.dir/layers/cuda_compile_generated_hdf5_data_layer.cu.o] Error 1 CMake Error at cuda_compile_generated_lrn_layer.cu.o.cmake:203 (message): Error generating /opt/caffe-jacinto/caffe-jacinto/build/src/caffe/CMakeFiles/cuda_compile.dir/layers/./cuda_compile_generated_lrn_layer.cu.o

make[2]: *** [src/caffe/CMakeFiles/cuda_compile.dir/solvers/cuda_compile_generated_nesterov_solver.cu.o] Error 1

... ... ... make[2]: *** [src/caffe/CMakeFiles/cuda_compile.dir/layers/cuda_compile_generated_softmax_layer.cu.o] Error 1 src/caffe/CMakeFiles/caffe.dir/build.make:315: recipe for target 'src/caffe/CMakeFiles/cuda_compile.dir/layers/cuda_compile_generated_softmax_layer.cu.o' failed nvcc fatal : Unsupported gpu architecture 'compute_70' CMake Error at cuda_compile_generated_sigmoid_cross_entropy_loss_layer.cu.o.cmake:203 (message): Error generating /opt/caffe-jacinto/caffe-jacinto/build/src/caffe/CMakeFiles/cuda_compile.dir/layers/./cuda_compile_generated_sigmoid_cross_entropy_loss_layer.cu.o

CMake Error at cuda_compile_generated_slice_layer.cu.o.cmake:203 (message): Error generating /opt/caffe-jacinto/caffe-jacinto/build/src/caffe/CMakeFiles/cuda_compile.dir/layers/./cuda_compile_generated_slice_layer.cu.o

src/caffe/CMakeFiles/caffe.dir/build.make:378: recipe for target 'src/caffe/CMakeFiles/cuda_compile.dir/layers/cuda_compile_generated_concat_layer.cu.o' failed make[2]: [src/caffe/CMakeFiles/cuda_compile.dir/layers/cuda_compile_generated_concat_layer.cu.o] Error 1 src/caffe/CMakeFiles/caffe.dir/build.make:455: recipe for target 'src/caffe/CMakeFiles/cuda_compile.dir/layers/cuda_compile_generated_inner_product_layer.cu.o' failed src/caffe/CMakeFiles/caffe.dir/build.make:364: recipe for target 'src/caffe/CMakeFiles/cuda_compile.dir/layers/cuda_compile_generated_sigmoid_cross_entropy_loss_layer.cu.o' failed make[2]: [src/caffe/CMakeFiles/cuda_compile.dir/layers/cuda_compile_generated_inner_product_layer.cu.o] Error 1 make[2]: [src/caffe/CMakeFiles/cuda_compile.dir/layers/cuda_compile_generated_sigmoid_cross_entropy_loss_layer.cu.o] Error 1 src/caffe/CMakeFiles/caffe.dir/build.make:413: recipe for target 'src/caffe/CMakeFiles/cuda_compile.dir/layers/cuda_compile_generated_slice_layer.cu.o' failed make[2]: [src/caffe/CMakeFiles/cuda_compile.dir/layers/cuda_compile_generated_slice_layer.cu.o] Error 1 make[1]: [src/caffe/CMakeFiles/caffe.dir/all] Error 2 CMakeFiles/Makefile2:272: recipe for target 'src/caffe/CMakeFiles/caffe.dir/all' failed make: [all] Error 2 Makefile:127: recipe for target 'all' failed The command '/bin/sh -c pip install --upgrade pip && cd caffe-jacinto/python && for req in $(cat requirements.txt) pydot; do pip install $req; done && cd .. && git clone https://github.com/NVIDIA/nccl.git && cd nccl && make -j install && cd .. && rm -rf nccl && mkdir build && cd build && cmake -DUSE_CUDNN=1 -DUSE_NCCL=1 .. && make -j"$(nproc)"' returned a non-zero code: 2

Thank you, Best, Yassine

Wronskia commented 7 years ago

The compute_70 problem seems to be supported only by cuda 9.0, to solve the problem, you just have to remove the 70 from du cmake/Cuda.cmake file https://github.com/tidsp/caffe-jacinto/blob/caffe-0.16/cmake/Cuda.cmake#L7 and related fields

Best, Yassine

mathmanu commented 7 years ago

Does the same issue exist in https://github.com/NVIDIA/caffe branch caffe-0.16? If so the the issue needs to be filed there.

Wronskia commented 7 years ago

I just tested and indeed the same issue exists in https://github.com/NVIDIA/caffe branch caffe-0.16. it is just that cuda 9.0 isn't available yet as a docker image and for those who are using cuda 8.0 or less, removing the 70 from cmake/Cuda.cmake solved the problem for me.

Thank you, Best, Yassine

mathmanu commented 7 years ago

Do you want to file an issue with NVIDIA/caffe or would you wait for cuda 9.0 to be available as a docker image?

Wronskia commented 7 years ago

Hello,

I am going to use your original version when cuda 9.0 will be available. Meanwhile, I will stick with cuda 8.0 without compute_70.

Best