search

facebookarchive / caffe2

Caffe2 is a lightweight, modular, and scalable deep learning framework.
https://caffe2.ai
Apache License 2.0
8.42k stars 1.95k forks source link

CUDNN_DATA_INT32 is undefined #1388

Open encore2020 opened 6 years ago

encore2020 commented 6 years ago

ubuntu@ubuntu-Z270N-WIFI:~/caffe2$ make && cd build && sudo make install -- Setting CMAKE_FIND_NO_INSTALL_PREFIX -- Using protobuf compiler /usr/bin/protoc -- The BLAS backend of choice:Eigen -- Could NOT find NNPACK (missing: NNPACK_INCLUDE_DIR NNPACK_LIBRARY PTHREADPOOL_LIBRARY) -- Will try to build NNPACK from source. If anything fails, follow the NNPACK prerequisite installation steps. CMake Warning at cmake/External/nnpack.cmake:155 (message): NNPACK is chosen to be installed, but confu and ninja that are needed by it are not installed. As a result we won't build with NNPACK. Call Stack (most recent call first): cmake/Dependencies.cmake:73 (include) CMakeLists.txt:74 (include)

CMake Warning at cmake/Dependencies.cmake:78 (message): Not compiling with NNPACK. Suppress this warning with -DUSE_NNPACK=OFF Call Stack (most recent call first): CMakeLists.txt:74 (include)

-- Found gflags (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libgflags.so) -- Found system gflags install. -- Found glog (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libglog.so) -- Found system glog install. -- Could NOT find Benchmark (missing: Benchmark_INCLUDE_DIR Benchmark_LIBRARY) -- git Version: v0.0.0 -- Version: 0.0.0 -- Performing Test HAVE_STD_REGEX -- Performing Test HAVE_STD_REGEX -- success -- Performing Test HAVE_GNU_POSIX_REGEX -- Performing Test HAVE_GNU_POSIX_REGEX -- failed to compile -- Performing Test HAVE_POSIX_REGEX -- Performing Test HAVE_POSIX_REGEX -- success -- Performing Test HAVE_STEADY_CLOCK -- Performing Test HAVE_STEADY_CLOCK -- success -- Found lmdb (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/liblmdb.so) -- Found LevelDB (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libleveldb.so) -- Found Snappy (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libsnappy.so) -- Could NOT find RocksDB (missing: RocksDB_INCLUDE_DIR RocksDB_LIBRARIES) CMake Warning at cmake/Dependencies.cmake:159 (message): Not compiling with RocksDB. Suppress this warning with -DUSE_ROCKSDB=OFF Call Stack (most recent call first): CMakeLists.txt:74 (include)

CMake Error at /usr/share/cmake-3.5/Modules/FindCUDA.cmake:617 (message): Specify CUDA_TOOLKIT_ROOT_DIR Call Stack (most recent call first): /usr/local/share/OpenCV/OpenCVConfig.cmake:67 (find_package) /usr/local/share/OpenCV/OpenCVConfig.cmake:86 (find_host_package) cmake/Dependencies.cmake:192 (find_package) CMakeLists.txt:74 (include)

-- Configuring incomplete, errors occurred! See also "/home/ubuntu/caffe2/build/CMakeFiles/CMakeOutput.log". See also "/home/ubuntu/caffe2/build/CMakeFiles/CMakeError.log". Makefile:4: recipe for target 'all' failed make: *** [all] Error 1 ubuntu@ubuntu-Z270N-WIFI:~/caffe2$

my configuration is ubuntu16+cuda9.0+gtx1080+cudnn7 how could I resolve the issue?

Maratyszcza commented 6 years ago

Specify CUDA_TOOLKIT_ROOT_DIR as suggested in the error message

encore2020 commented 6 years ago

could tell me how to do that?

how to Specify CUDA_TOOLKIT_ROOT_DIR as suggested in the error message

encore2020 commented 6 years ago

ubuntu@ubuntu-Z270N-WIFI:~/caffe2$ make && cd build && sudo make install -- Setting CMAKE_FIND_NO_INSTALL_PREFIX -- Using protobuf compiler /usr/bin/protoc -- The BLAS backend of choice:Eigen -- Could NOT find NNPACK (missing: NNPACK_INCLUDE_DIR NNPACK_LIBRARY PTHREADPOOL_LIBRARY) -- Will try to build NNPACK from source. If anything fails, follow the NNPACK prerequisite installation steps. CMake Warning at cmake/External/nnpack.cmake:155 (message): NNPACK is chosen to be installed, but confu and ninja that are needed by it are not installed. As a result we won't build with NNPACK. Call Stack (most recent call first): cmake/Dependencies.cmake:73 (include) CMakeLists.txt:74 (include)

CMake Warning at cmake/Dependencies.cmake:78 (message): Not compiling with NNPACK. Suppress this warning with -DUSE_NNPACK=OFF Call Stack (most recent call first): CMakeLists.txt:74 (include)

-- Found gflags (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libgflags.so) -- Found system gflags install. -- Found glog (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libglog.so) -- Found system glog install. -- Could NOT find Benchmark (missing: Benchmark_INCLUDE_DIR Benchmark_LIBRARY) -- git Version: v0.0.0 -- Version: 0.0.0 -- Performing Test HAVE_STD_REGEX -- Performing Test HAVE_STD_REGEX -- success -- Performing Test HAVE_GNU_POSIX_REGEX -- Performing Test HAVE_GNU_POSIX_REGEX -- failed to compile -- Performing Test HAVE_POSIX_REGEX -- Performing Test HAVE_POSIX_REGEX -- success -- Performing Test HAVE_STEADY_CLOCK -- Performing Test HAVE_STEADY_CLOCK -- success -- Found lmdb (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/liblmdb.so) -- Found LevelDB (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libleveldb.so) -- Found Snappy (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libsnappy.so) -- Could NOT find RocksDB (missing: RocksDB_INCLUDE_DIR RocksDB_LIBRARIES) CMake Warning at cmake/Dependencies.cmake:159 (message): Not compiling with RocksDB. Suppress this warning with -DUSE_ROCKSDB=OFF Call Stack (most recent call first): CMakeLists.txt:74 (include)

CMake Error at /usr/share/cmake-3.5/Modules/FindPackageHandleStandardArgs.cmake:148 (message): Could NOT find CUDA: Found unsuitable version "9.0", but required is exact version "8.0" (found /usr/local/cuda-9.0) Call Stack (most recent call first): /usr/share/cmake-3.5/Modules/FindPackageHandleStandardArgs.cmake:386 (_FPHSA_FAILURE_MESSAGE) /usr/share/cmake-3.5/Modules/FindCUDA.cmake:949 (find_package_handle_standard_args) /usr/local/share/OpenCV/OpenCVConfig.cmake:67 (find_package) /usr/local/share/OpenCV/OpenCVConfig.cmake:86 (find_host_package) cmake/Dependencies.cmake:192 (find_package) CMakeLists.txt:74 (include)

-- Configuring incomplete, errors occurred! See also "/home/ubuntu/caffe2/build/CMakeFiles/CMakeOutput.log". See also "/home/ubuntu/caffe2/build/CMakeFiles/CMakeError.log". Makefile:4: recipe for target 'all' failed make: *** [all] Error 1 ubuntu@ubuntu-Z270N-WIFI:~/caffe2$

Does it mean that my cuda 9.0 could not be used?

AurusHuang commented 6 years ago

My suggestions: First, you must have a CUDA-compatible GPU. Second, try to uninstall your all CUDA versions, and install CUDA 9.0 only. Perform tests recommended by NVIDIA CUDA install guide (a pdf file from CUDA official site) to check if your CUDA works properly. Third, check your GCC version. If your GCC version is 6.0 or higher, try my workaround on #1356 . Finally, make sure you're using a new version of source code. Old source codes may have errors and bugs. I have successfully built all Caffe2 libraries and auxiliary programs under Ubuntu 17.04+CUDA 9.0.176+GCC 6.0.

encore2020 commented 6 years ago

I have re-installed cuda 9.0 , cuda test is ok

Then about cudnn ,how could I installed it? Only cudnn v7 is available for cuda 9.0. Then how to install it?

I have installed it by, sudo dpkg -i libcudnn7_7.0.3.11-1+cuda9.0_amd64.deb sudo dpkg -i libcudnn7-dev_7.0.3.11-1+cuda9.0_amd64.deb

but the result is not right

57%] Building NVCC (Device) object caffe2/CMakeFiles/caffe2_gpu.dir/utils/caffe2_gpu_generated_math_gpu.cu.o /home/ubuntu/caffe2/caffe2/core/common_cudnn.h(31): error: static assertion failed with "Caffe2 requires cudnn version 5.0 or above."

/home/ubuntu/caffe2/caffe2/core/common_cudnn.h(142): error: identifier "CUDNN_DATA_INT32" is undefined

/home/ubuntu/caffe2/caffe2/core/common_cudnn.h(288): error: too many arguments in function call

3 errors detected in the compilation of "/tmp/tmpxft_000010d4_00000000-12_math_gpu.compute_70.cpp1.ii". CMake Error at caffe2_gpu_generated_math_gpu.cu.o.cmake:262 (message): Error generating file /home/ubuntu/caffe2/build/caffe2/CMakeFiles/caffe2_gpu.dir/utils/./caffe2_gpu_generated_math_gpu.cu.o

caffe2/CMakeFiles/caffe2_gpu.dir/build.make:1154: recipe for target 'caffe2/CMakeFiles/caffe2_gpu.dir/utils/caffe2_gpu_generated_math_gpu.cu.o' failed make[3]: [caffe2/CMakeFiles/caffe2_gpu.dir/utils/caffe2_gpu_generated_math_gpu.cu.o] Error 1 make[3]: Leaving directory '/home/ubuntu/caffe2/build' CMakeFiles/Makefile2:1986: recipe for target 'caffe2/CMakeFiles/caffe2_gpu.dir/all' failed make[2]: [caffe2/CMakeFiles/caffe2_gpu.dir/all] Error 2 make[2]: Leaving directory '/home/ubuntu/caffe2/build' Makefile:138: recipe for target 'all' failed make[1]: [all] Error 2 make[1]: Leaving directory '/home/ubuntu/caffe2/build' Makefile:4: recipe for target 'all' failed make: [all] Error 2 ubuntu@ubuntu-Z270N-WIFI:~/caffe2$

fbadaud commented 6 years ago

hello I got similar initial issue about the NNPACK not found: Could NOT find NNPACK (missing: NNPACK_INCLUDE_DIR NNPACK_LIBRARY PTHREADPOOL_LIBRARY) when recompiling caffe2 after a today git pull.

I have tried to install NNPACK but not succeed with the peachPy and confu installation following the Maratyszcza/NNPACK page.

For now I set to off the use of NNPACK in the CMakelist.txt. Will it have a big impact if I am not using CUDA?

secondly at about 71% of compilation, I run in another new issue with pybind11:

In file included from /home/b2bot/Project/caffe2/caffe2/caffe2/python/pybind_state.cc:17:0: /home/b2bot/Project/caffe2/caffe2/caffe2/python/pybind_state.h: In member function ‘caffe2::python::BlobFetcherBase::FetchedBlob caffe2::python::TensorFetcher::FetchTensor(const caffe2::Tensor&, bool)’: /home/b2bot/Project/caffe2/caffe2/caffe2/python/pybind_state.h:133:20: error: ‘reinterpret_steal’ is not a member of ‘caffe2::python::py’ result.obj = py::reinterpret_steal( ^ /home/b2bot/Project/caffe2/caffe2/caffe2/python/pybind_state.h:133:52: error: expected primary-expression before ‘>’ token result.obj = py::reinterpret_steal( ^ /home/b2bot/Project/caffe2/caffe2/caffe2/python/pybind_state.h:139:20: error: ‘reinterpret_steal’ is not a member of ‘caffe2::python::py’ result.obj = py::reinterpret_steal(PyArray_SimpleNewFromData( ^ /home/b2bot/Project/caffe2/caffe2/caffe2/python/pybind_state.h:139:52: error: expected primary-expression before ‘>’ token result.obj = py::reinterpret_steal(PyArray_SimpleNewFromData( ^ /home/b2bot/Project/caffe2/caffe2/caffe2/python/pybind_state.cc: In constructor ‘caffe2::python::PythonOpBase::PythonOpBase(const caffe2::OperatorDef&, caffe2::Workspace*, const string&)’: /home/b2bot/Project/caffe2/caffe2/caffe2/python/pybind_state.cc:254:11: error: ‘reinterpret_steal’ is not a member of ‘caffe2::python::py’ py::reinterpret_steal(PyImport_ImportModule("pickle")); ^ /home/b2bot/Project/caffe2/caffe2/caffe2/python/pybind_state.cc:254:43: error: expected primary-expression before ‘>’ token py::reinterpret_steal(PyImport_ImportModule("pickle")); ^ /home/b2bot/Project/caffe2/caffe2/caffe2/python/pybind_state.cc:256:56: error: expected primary-expression before ‘>’ token auto loads = pickle.attr("loads").cast();

thanks and BR Francois

fbadaud commented 6 years ago

good morning I have seen that some updates have been made to the python/pybind_state.h etc... 4 days ago. in my caffe2 installation the third_party/pybind11/ is dated from my first installation on 28 july 17 . Could I have an issue with a missing recompilation of the lib? if yes how can I recompile it with its latest version for caffe2? thanks in advance Francois

fbadaud commented 6 years ago

I solve the both compilation issues with pybind11 and NNPACK by installing tools on my disk from their various path: https://github.com/pybind/pybind11/releases https://github.com/Maratyszcza/NNPACK

and create a link to them inside the caffe2/third_party/ directory I hope it will help others. Best Regards francois

pietern commented 6 years ago

@encore2020 Did you confirm in the CMake output that you were compiling against cuDNN 7?

@fbadaud Your issue is unrelated to the original one, if you have other issues in the future please feel free to open a separate issue. I think the original cause for your issue was an old pybind11 version installed on your system.