search

NVIDIA / DIGITS

Deep Learning GPU Training System
https://developer.nvidia.com/digits
BSD 3-Clause "New" or "Revised" License
4.12k stars 1.38k forks source link

Building Caffe: unset variable CUDA_cublas_device_LIBRARY breaks CMake build configuration #2185

Open monocongo opened 5 years ago

monocongo commented 5 years ago

I am trying to build Caffe as one of the prerequisites for DIGITS and at the step where I use CMake to configure the build:

$ cmake ..    
-- Boost version: 1.65.1
-- Found the following Boost libraries:
--   system
--   thread
--   filesystem
--   chrono
--   date_time
--   atomic
-- Found gflags  (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libgflags.so)
-- Found glog    (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libglog.so)
-- Found PROTOBUF Compiler: /usr/local/bin/protoc
-- HDF5: Using hdf5 compiler wrapper to determine C configuration
-- HDF5: Using hdf5 compiler wrapper to determine CXX configuration
-- Found lmdb    (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/liblmdb.so)
-- Found LevelDB (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libleveldb.so)
-- Found Snappy  (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libsnappy.so)
-- CUDA detected: 10.0
-- Found cuDNN: ver. 7.6.0 found (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libcudnn.so)
-- Added CUDA NVCC flags for: sm_61
-- OpenCV found (/usr/share/OpenCV)
-- Found Atlas (include: /usr/include/x86_64-linux-gnu, library: /usr/lib/x86_64-linux-gnu/libatlas.so)
-- NumPy ver. 1.13.3 found (include: /usr/lib/python2.7/dist-packages/numpy/core/include)
-- Boost version: 1.65.1
-- Found the following Boost libraries:
--   python
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) 
-- Could NOT find NCCL (missing: NCCL_INCLUDE_DIR NCCL_LIBRARY) 
-- 
-- ******************* Caffe Configuration Summary *******************
-- General:
--   Version           :   0.15.14
--   Git               :   v0.15.14-16-g4b8d54d8
--   System            :   Linux
--   C++ compiler      :   /usr/bin/c++
--   Release CXX flags :   -O3 -DNDEBUG -fPIC -Wall -Wno-sign-compare -Wno-uninitialized
--   Debug CXX flags   :   -g -fPIC -Wall -Wno-sign-compare -Wno-uninitialized
--   Build type        :   Release
-- 
--   BUILD_SHARED_LIBS :   ON
--   BUILD_python      :   ON
--   BUILD_matlab      :   OFF
--   BUILD_docs        :   ON
--   CPU_ONLY          :   OFF
--   USE_OPENCV        :   ON
--   USE_LEVELDB       :   ON
--   USE_LMDB          :   ON
--   ALLOW_LMDB_NOLOCK :   OFF
-- 
-- Dependencies:
--   BLAS              :   Yes (Atlas)
--   Boost             :   Yes (ver. 1.65)
--   glog              :   Yes
--   gflags            :   Yes
--   protobuf          :   Yes (ver. 3.2.0)
--   lmdb              :   Yes (ver. 0.9.21)
--   LevelDB           :   Yes (ver. 1.20)
--   Snappy            :   Yes (ver. ..)
--   OpenCV            :   Yes (ver. 3.2.0)
--   CUDA              :   Yes (ver. 10.0)
-- 
-- NVIDIA CUDA:
--   Target GPU(s)     :   Auto
--   GPU arch(s)       :   sm_61
--   cuDNN             :   Yes (ver. 7.6.0)
--   NCCL              :   Not found
-- 
-- Python:
--   Interpreter       :   /usr/bin/python2.7 (ver. 2.7.15)
--   Libraries         :   /usr/lib/x86_64-linux-gnu/libpython2.7.so (ver 2.7.15+)
--   NumPy             :   /usr/lib/python2.7/dist-packages/numpy/core/include (ver 1.13.3)
-- 
-- Documentaion:
--   Doxygen           :   No
--   config_file       :   
-- 
-- Install:
--   Install path      :   /home/james/git/caffe/build/install
-- 
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
CUDA_cublas_device_LIBRARY (ADVANCED)
    linked by target "caffe" in directory /home/james/git/caffe/src/caffe

-- Configuring incomplete, errors occurred!
See also "/home/james/git/caffe/build/CMakeFiles/CMakeOutput.log".
See also "/home/james/git/caffe/build/CMakeFiles/CMakeError.log".

Can someone advise as to how I can get past this issue? Thanks in advance for any assistance.

Olyseus commented 5 years ago

CUBLAS is apparently not installed on your system. Or it could be related to this issue, i.e. some old version of CMake. https://stackoverflow.com/questions/54347530/how-to-link-cublas-library-with-cmake-cuda-10-0-ubuntu-18

Antoine Cottin, CEO — Olyseus Innovations Ltd.  Kimonos 41, Agios Nektarios, CY-3095 Limassol, Cyprus +357-99-88-90-79

From: James Adams Sent: Wednesday, August 28, 2019 1:18 AM To: NVIDIA/DIGITS Cc: Subscribed Subject: [NVIDIA/DIGITS] Building Caffe: unset variableCUDA_cublas_device_LIBRARY breaks CMake build configuration (#2185)

I am trying to build Caffe as one of the prerequisites for DIGITS and at the step where I use CMake to configure the build: $ cmake ..
-- Boost version: 1.65.1 -- Found the following Boost libraries: -- system -- thread -- filesystem -- chrono -- date_time -- atomic -- Found gflags (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libgflags.so) -- Found glog (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libglog.so) -- Found PROTOBUF Compiler: /usr/local/bin/protoc -- HDF5: Using hdf5 compiler wrapper to determine C configuration -- HDF5: Using hdf5 compiler wrapper to determine CXX configuration -- Found lmdb (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/liblmdb.so) -- Found LevelDB (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libleveldb.so) -- Found Snappy (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libsnappy.so) -- CUDA detected: 10.0 -- Found cuDNN: ver. 7.6.0 found (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libcudnn.so) -- Added CUDA NVCC flags for: sm_61 -- OpenCV found (/usr/share/OpenCV) -- Found Atlas (include: /usr/include/x86_64-linux-gnu, library: /usr/lib/x86_64-linux-gnu/libatlas.so) -- NumPy ver. 1.13.3 found (include: /usr/lib/python2.7/dist-packages/numpy/core/include) -- Boost version: 1.65.1 -- Found the following Boost libraries: -- python -- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) -- Could NOT find NCCL (missing: NCCL_INCLUDE_DIR NCCL_LIBRARY) -- -- Caffe Configuration Summary -- General: -- Version : 0.15.14 -- Git : v0.15.14-16-g4b8d54d8 -- System : Linux -- C++ compiler : /usr/bin/c++ -- Release CXX flags : -O3 -DNDEBUG -fPIC -Wall -Wno-sign-compare -Wno-uninitialized -- Debug CXX flags : -g -fPIC -Wall -Wno-sign-compare -Wno-uninitialized -- Build type : Release -- -- BUILD_SHARED_LIBS : ON -- BUILD_python : ON -- BUILD_matlab : OFF -- BUILD_docs : ON -- CPU_ONLY : OFF -- USE_OPENCV : ON -- USE_LEVELDB : ON -- USE_LMDB : ON -- ALLOW_LMDB_NOLOCK : OFF -- -- Dependencies: -- BLAS : Yes (Atlas) -- Boost : Yes (ver. 1.65) -- glog : Yes -- gflags : Yes -- protobuf : Yes (ver. 3.2.0) -- lmdb : Yes (ver. 0.9.21) -- LevelDB : Yes (ver. 1.20) -- Snappy : Yes (ver. ..) -- OpenCV : Yes (ver. 3.2.0) -- CUDA : Yes (ver. 10.0) -- -- NVIDIA CUDA: -- Target GPU(s) : Auto -- GPU arch(s) : sm_61 -- cuDNN : Yes (ver. 7.6.0) -- NCCL : Not found -- -- Python: -- Interpreter : /usr/bin/python2.7 (ver. 2.7.15) -- Libraries : /usr/lib/x86_64-linux-gnu/libpython2.7.so (ver 2.7.15+) -- NumPy : /usr/lib/python2.7/dist-packages/numpy/core/include (ver 1.13.3) -- -- Documentaion: -- Doxygen : No -- config_file :
-- -- Install: -- Install path : /home/james/git/caffe/build/install -- CMake Error: The following variables are used in this project, but they are set to NOTFOUND. Please set them or make sure they are set and tested correctly in the CMake files: CUDA_cublas_device_LIBRARY (ADVANCED) linked by target "caffe" in directory /home/james/git/caffe/src/caffe

-- Configuring incomplete, errors occurred! See also "/home/james/git/caffe/build/CMakeFiles/CMakeOutput.log". See also "/home/james/git/caffe/build/CMakeFiles/CMakeError.log". Can someone advise as to how I can get past this issue? Thanks in advance for any assistance. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

monocongo commented 5 years ago

Thanks, @Olyseus. Is CUBLAS supposed to be included as part of the CUDA installation, or is it normal to need to install it separately? This is where I should install it from, correct?

monocongo commented 5 years ago

I followed these instructions to reinstall CUDA 10.1, as well as sudo apt autoremove that removed CUDA 10.0. I updated my CUDA_HOME to /usr/local/cuda-10.1 and now the CMake configuration doesn't even find CUDA (and the CUBLAS libraries don't appear under /usr/local/cuda-10.1/lib64):

$ cmake ..                  
-- Boost version: 1.65.1
-- Found the following Boost libraries:
--   system
--   thread
--   filesystem
--   chrono
--   date_time
--   atomic
-- Found gflags  (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libgflags.so)
-- Found glog    (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libglog.so)
-- Found PROTOBUF Compiler: /usr/local/bin/protoc
-- HDF5: Using hdf5 compiler wrapper to determine C configuration
-- HDF5: Using hdf5 compiler wrapper to determine CXX configuration
-- Found lmdb    (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/liblmdb.so)
-- Found LevelDB (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libleveldb.so)
-- Found Snappy  (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libsnappy.so)
CMake Warning at cmake/Dependencies.cmake:63 (message):
  -- CUDA is not detected by cmake.  Building without it...
Call Stack (most recent call first):
  CMakeLists.txt:58 (include)

-- OpenCV found (/usr/share/OpenCV)
-- Found Atlas (include: /usr/include/x86_64-linux-gnu, library: /usr/lib/x86_64-linux-gnu/libatlas.so)
-- NumPy ver. 1.13.3 found (include: /usr/lib/python2.7/dist-packages/numpy/core/include)
-- Boost version: 1.65.1
-- Found the following Boost libraries:
--   python
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) 
-- Could NOT find NCCL (missing: NCCL_INCLUDE_DIR NCCL_LIBRARY) 
-- 
-- ******************* Caffe Configuration Summary *******************
-- General:
--   Version           :   0.15.14
--   Git               :   v0.15.14-16-g4b8d54d8
--   System            :   Linux
--   C++ compiler      :   /usr/bin/c++
--   Release CXX flags :   -O3 -DNDEBUG -fPIC -Wall -Wno-sign-compare -Wno-uninitialized
--   Debug CXX flags   :   -g -fPIC -Wall -Wno-sign-compare -Wno-uninitialized
--   Build type        :   Release
-- 
--   BUILD_SHARED_LIBS :   ON
--   BUILD_python      :   ON
--   BUILD_matlab      :   OFF
--   BUILD_docs        :   ON
--   CPU_ONLY          :   OFF
--   USE_OPENCV        :   ON
--   USE_LEVELDB       :   ON
--   USE_LMDB          :   ON
--   ALLOW_LMDB_NOLOCK :   OFF
-- 
-- Dependencies:
--   BLAS              :   Yes (Atlas)
--   Boost             :   Yes (ver. 1.65)
--   glog              :   Yes
--   gflags            :   Yes
--   protobuf          :   Yes (ver. 3.2.0)
--   lmdb              :   Yes (ver. 0.9.21)
--   LevelDB           :   Yes (ver. 1.20)
--   Snappy            :   Yes (ver. ..)
--   OpenCV            :   Yes (ver. 3.2.0)
--   CUDA              :   No
-- 
-- Python:
--   Interpreter       :   /usr/bin/python2.7 (ver. 2.7.15)
--   Libraries         :   /usr/lib/x86_64-linux-gnu/libpython2.7.so (ver 2.7.15+)
--   NumPy             :   /usr/lib/python2.7/dist-packages/numpy/core/include (ver 1.13.3)
-- 
-- Documentaion:
--   Doxygen           :   No
--   config_file       :   
-- 
-- Install:
--   Install path      :   /home/james/git/caffe/build/install
-- 
-- Configuring done
-- Generating done
-- Build files have been written to: /home/james/git/caffe/build

BTW my CMake is at version 3.10.2 which appears to be the up-to-date version for Ubuntu 18.04.

evil-potato commented 4 years ago

Hi，I also have the same problem. Is there any way to solve it?

starlingonearth commented 4 weeks ago

using a newest version of cmake ,such as camke 3.30.5,it works.