Closed engineer1109 closed 3 years ago
my build log is here: http://phd-sid.ethz.ch/debian/caffe-cuda/caffe-contrib_1.0.0-8ubuntu2_amd64.build
I ran into this myself, and my fix was to upgrade my CMake version to 3.12.2, I got the idea from: https://github.com/clab/dynet/issues/1457
I dug into it, and the CUDA_cublas_device_LIBRARY
cmake macro is defined in a fairly recent version of CMake (not sure if 3.12.2 introduced it, but at least it contains it).
I hope that helps!
I got it to build from source, no problem. Thanks...
Also facing this problem building caffe from source on centos 7. I installed cmake 3.6.2.
[UruGN@urugn-lap build]$ cmake -DBLAS=open .. Boost version: 1.58.0 Found the following Boost libraries: system thread filesystem chrono date_time atomic Found gflags (include: /usr/include, library: /usr/lib64/libgflags.so) Found glog (include: /usr/include, library: /usr/lib64/libglog.so) Found PROTOBUF Compiler: /usr/bin/protoc HDF5: Using hdf5 compiler wrapper to determine C configuration HDF5: Using hdf5 compiler wrapper to determine CXX configuration Found lmdb (include: /usr/include, library: /usr/lib64/liblmdb.so) Found LevelDB (include: /usr/include, library: /usr/lib64/libleveldb.so) Found Snappy (include: /usr/include, library: /usr/lib64/libsnappy.so) CUDA detected: 10.1 Automatic GPU detection failed. Building for all known architectures. Added CUDA NVCC flags for: sm_20 sm_21 sm_30 sm_35 sm_50 sm_60 sm_61 OpenCV found (/usr/lib64/cmake/opencv4) Found OpenBLAS libraries: /usr/lib64/libopenblas.so Found OpenBLAS include: /usr/include NumPy ver. 1.7.1 found (include: /usr/lib64/python2.7/site-packages/numpy/core/include) Boost version: 1.58.0 Found the following Boost libraries: python Detected Doxygen OUTPUT_DIRECTORY: ./doxygen/
Caffe Configuration Summary General: Version : 1.0.0 Git : 1.0-134-g04ab089 System : Linux C++ compiler : /usr/bin/c++ Release CXX flags : -O3 -DNDEBUG -fPIC -Wall -Wno-sign-compare -Wno-uninitialized Debug CXX flags : -g -fPIC -Wall -Wno-sign-compare -Wno-uninitialized Build type : Release
BUILD_SHARED_LIBS : ON BUILD_python : ON BUILD_matlab : OFF BUILD_docs : ON CPU_ONLY : OFF USE_OPENCV : ON USE_LEVELDB : ON USE_LMDB : ON USE_NCCL : OFF ALLOW_LMDB_NOLOCK : OFF USE_HDF5 : ON
Dependencies: BLAS : Yes (open) Boost : Yes (ver. 1.58) glog : Yes gflags : Yes protobuf : Yes (ver. ..) lmdb : Yes (ver. 0.9.22) LevelDB : Yes (ver. 1.12) Snappy : Yes (ver. 1.1.0) OpenCV : Yes (ver. 4.0.1) CUDA : Yes (ver. 10.1)
NVIDIA CUDA: Target GPU(s) : Auto GPU arch(s) : sm_20 sm_21 sm_30 sm_35 sm_50 sm_60 sm_61 cuDNN : Not found
Python: Interpreter : /usr/bin/python2.7 (ver. 2.7.5) Libraries : /usr/lib64/libpython2.7.so (ver 2.7.5) NumPy : /usr/lib64/python2.7/site-packages/numpy/core/include (ver 1.7.1)
Documentaion: Doxygen : /usr/bin/doxygen (1.8.5) config_file : /developer/resources/ai/caffe/caffe/.Doxyfile
Install: Install path : /developer/resources/ai/caffe/caffe/build/install
CMake Error: The following variables are used in this project, but they are set to NOTFOUND. Please set them or make sure they are set and tested correctly in the CMake files: CUDA_cublas_device_LIBRARY (ADVANCED) linked by target "caffe" in directory /developer/resources/ai/caffe/caffe/src/caffe
Configuring incomplete, errors occurred! See also "/developer/resources/ai/caffe/caffe/build/CMakeFiles/CMakeOutput.log". See also "/developer/resources/ai/caffe/caffe/build/CMakeFiles/CMakeError.log".
CMakeError.log
Determining if the pthread_create exist failed with the following output: Change Dir: /developer/resources/ai/caffe/caffe/build/CMakeFiles/CMakeTmp
Run Build Command:"/usr/bin/gmake" "cmTC_0a241/fast"
/usr/bin/gmake -f CMakeFiles/cmTC_0a241.dir/build.make CMakeFiles/cmTC_0a241.dir/build
gmake[1]: Entering directory /urugn/developer/resources/ai/caffe/caffe/build/CMakeFiles/CMakeTmp' Building C object CMakeFiles/cmTC_0a241.dir/CheckSymbolExists.c.o /usr/bin/cc -o CMakeFiles/cmTC_0a241.dir/CheckSymbolExists.c.o -c /developer/resources/ai/caffe/caffe/build/CMakeFiles/CMakeTmp/CheckSymbolExists.c Linking C executable cmTC_0a241 /usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_0a241.dir/link.txt --verbose=1 /usr/bin/cc CMakeFiles/cmTC_0a241.dir/CheckSymbolExists.c.o -o cmTC_0a241 -rdynamic CMakeFiles/cmTC_0a241.dir/CheckSymbolExists.c.o: In function
main':
CheckSymbolExists.c:(.text+0x16): undefined reference to pthread_create' collect2: error: ld returned 1 exit status gmake[1]: *** [cmTC_0a241] Error 1 gmake[1]: Leaving directory
/urugn/developer/resources/ai/caffe/caffe/build/CMakeFiles/CMakeTmp'
gmake: *** [cmTC_0a241/fast] Error 2
File /developer/resources/ai/caffe/caffe/build/CMakeFiles/CMakeTmp/CheckSymbolExists.c: / /
int main(int argc, char** argv) { (void)argv;
return ((int*)(&pthread_create))[argc];
(void)argc; return 0;
}
Determining if the function pthread_create exists in the pthreads failed with the following output: Change Dir: /developer/resources/ai/caffe/caffe/build/CMakeFiles/CMakeTmp
Run Build Command:"/usr/bin/gmake" "cmTC_bb6d5/fast"
/usr/bin/gmake -f CMakeFiles/cmTC_bb6d5.dir/build.make CMakeFiles/cmTC_bb6d5.dir/build
gmake[1]: Entering directory /urugn/developer/resources/ai/caffe/caffe/build/CMakeFiles/CMakeTmp' Building C object CMakeFiles/cmTC_bb6d5.dir/CheckFunctionExists.c.o /usr/bin/cc -DCHECK_FUNCTION_EXISTS=pthread_create -o CMakeFiles/cmTC_bb6d5.dir/CheckFunctionExists.c.o -c /usr/share/cmake-3.6/Modules/CheckFunctionExists.c Linking C executable cmTC_bb6d5 /usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_bb6d5.dir/link.txt --verbose=1 /usr/bin/cc -DCHECK_FUNCTION_EXISTS=pthread_create CMakeFiles/cmTC_bb6d5.dir/CheckFunctionExists.c.o -o cmTC_bb6d5 -rdynamic -lpthreads /usr/bin/ld: cannot find -lpthreads collect2: error: ld returned 1 exit status gmake[1]: *** [cmTC_bb6d5] Error 1 gmake[1]: Leaving directory
/urugn/developer/resources/ai/caffe/caffe/build/CMakeFiles/CMakeTmp'
gmake: *** [cmTC_bb6d5/fast] Error 2
Yet i have pthread and pthreads installed. I put the following lines in CMakeLists.txt
cmake_minimum_required (VERSION 2.8.7) find_package(Threads)
Then i run cmake .
and i get the following
[UruGN@urugn-lap test]$ cmake . Looking for pthread.h Looking for pthread.h - found Looking for pthread_create Looking for pthread_create - not found Looking for pthread_create in pthreads Looking for pthread_create in pthreads - not found Looking for pthread_create in pthread Looking for pthread_create in pthread - found Found Threads: TRUE Configuring done Generating done Build files have been written to: /developer/resources/cmake/test
Also system reports pthread_create exists
nm /lib64/libpthread.so.0 | grep "pthread_create"
nm output
00000000000080e0 t __pthread_create_2_1
00000000000080e0 T pthread_create@@GLIBC_2.2.5
ldd which tar
| grep pthread
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f9040c9a000)
Any lead as to what am missing?
After installing cmake 3.12.4 and also installing cuDNN library and the devel package i still had the create_pthread issue. But after i deleted the build directory and recreated it, am now past the create_pthread issue and cmake -DBLAS=open ..
completes without error. Now am having c++ compiler error when i run make.
` In file included from /usr/include/c++/4.8.2/atomic:38:0, from /usr/include/google/protobuf/io/coded_stream.h:115, from /developer/resources/ai/caffe/caffe/build/include/caffe/proto/caffe.pb.h:23, from /developer/resources/ai/caffe/caffe/build/include/caffe/proto/caffe.pb.cc:4: /usr/include/c++/4.8.2/bits/c++0x_warning.h:32:2: error: #error This file requires compiler and library support for the ISO C++ 2011 standard. This support is currently experimental, and must be enabled with the -std=c++11 or -std=gnu++11 compiler options.
^
In file included from /usr/include/c++/4.8.2/atomic:41:0,
from /usr/include/google/protobuf/io/coded_stream.h:115,
from /developer/resources/ai/caffe/caffe/build/include/caffe/proto/caffe.pb.h:23,
from /developer/resources/ai/caffe/caffe/build/include/caffe/proto/caffe.pb.cc:4:
/usr/include/c++/4.8.2/bits/atomic_base.h:70:3: error: ‘constexpr’ does not name a type
constexpr memory_order
^
/usr/include/c++/4.8.2/bits/atomic_base.h:70:3: note: C++11 ‘constexpr’ only available with -std=c++11 or -std=gnu++11
/usr/include/c++/4.8.2/bits/atomic_base.h:76:3: error: ‘constexpr’ does not name a type
constexpr memory_order
^
/usr/include/c++/4.8.2/bits/atomic_base.h:76:3: note: C++11 ‘constexpr’ only available with -std=c++11 or -std=gnu++11
/usr/include/c++/4.8.2/bits/atomic_base.h:83:3: error: ‘constexpr’ does not name a type
constexpr memory_order
^
/usr/include/c++/4.8.2/bits/atomic_base.h:83:3: note: C++11 ‘constexpr’ only available with -std=c++11 or -std=gnu++11
/usr/include/c++/4.8.2/bits/atomic_base.h:90:3: error: ‘constexpr’ does not name a type
constexpr memory_order
^
/usr/include/c++/4.8.2/bits/atomic_base.h:90:3: note: C++11 ‘constexpr’ only available with -std=c++11 or -std=gnu++11
/usr/include/c++/4.8.2/bits/atomic_base.h:98:41: error: expected initializer before ‘noexcept’
atomic_thread_fence(memory_order m) noexcept
^
/usr/include/c++/4.8.2/bits/atomic_base.h:102:41: error: expected initializer before ‘noexcept’
atomic_signal_fence(memory_order m) noexcept
^
/usr/include/c++/4.8.2/bits/atomic_base.h:108:30: error: expected initializer before ‘noexcept’
kill_dependency(_Tp y) noexcept
^
/usr/include/c++/4.8.2/bits/atomic_base.h:156:25: error: ‘char16_t’ was not declared in this scope
typedef atomic_base
Any leads on what am i supposed to do. Perhaps enable c++11? Thx
After a week of war i finally got Caffe to link. Most importantly i had to compile most dependencies from source and with the same version of gcc in my case i used 7.3.0.
protobuf
./configure CXXFLAGS=-std=c++11 --prefix=/usr --with-cc=/usr/local/bin/gcc --with-cxx=/usr/local/bin/g++ --enable-shared --with-pic
gtest
cmake -D CMAKE_C_COMPILER=/usr/local/bin/gcc -D CMAKE_CXX_COMPILER=/usr/local/bin/g++ -D CMAKE_INSTALL_PREFIX=/usr -DBUILD_SHARED_LIBS=ON ..
leveldb,
cmake -DCMAKE_BUILD_TYPE=Release -D CMAKE_C_COMPILER=/usr/local/bin/gcc -D CMAKE_CXX_COMPILER=/usr/local/bin/g++ -D CMAKE_INSTALL_PREFIX=/usr -DBUILD_SHARED_LIBS=ON ..
gflags
cmake -D CMAKE_C_COMPILER=/usr/local/bin/gcc -D CMAKE_CXX_COMPILER=/usr/local/bin/g++ -D CMAKE_INSTALL_PREFIX=/usr -DBUILD_SHARED_LIBS=ON ..
glog
./autogen.sh && ./configure --prefix=/usr --enable-shared --with-pic
I had also to make some sym links for /usr/lib/libglog.so and /usr/lib/libgflags.so to /usr/lib64
hope this will helps future ML newbees..
I am facing the similar problem, i use cuda10.1 and ubuntu18.04, and i have also tried the cmake 3.12.2. Could you give me some solution steps ? much thanks!
CMake Error: The following variables are used in this project, but they are set to NOTFOUND. Please set them or make sure they are set and tested correctly in the CMake files: CUDA_cublas_LIBRARY (ADVANCED) linked by target "caffe" in directory /home/vilbert_beta/bottom-up-attention/caffe/src/caffe
-- Configuring incomplete, errors occurred! See also "/home/vilbert_beta/bottom-up-attention/caffe/build/CMakeFiles/CMakeOutput.log". See also "/home/vilbert_beta/bottom-up-attention/caffe/build/CMakeFiles/CMakeError.log".
the same to me .who can help me .thanks!
the same to me .who can help me .thanks!
well, this problem may occur when you using cmake in old version. As i mentioned above, cmake 3.12.2 also do not work. finally, i fix that through using cmake 3.14 in Anaconda. Try to upgrade your cmake version. Hope this helps you.
thanks . my cmake is 3.16.3 . i have done it through that set(CUDA_cublas_LIBRARY /usr/lib/x86_64-linux-gnu/libcublas.so.10 )
abandon the caffe for years
python2 cuda10 cudnn7 Ubuntu18.04 -- The C compiler identification is GNU 7.3.0 -- The CXX compiler identification is GNU 7.3.0 -- Check for working C compiler: /usr/bin/cc -- Check for working C compiler: /usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done CMake Warning at /usr/share/cmake-3.10/Modules/FindBoost.cmake:801 (message): New Boost version may have incorrect or missing dependencies and imported targets Call Stack (most recent call first): /usr/share/cmake-3.10/Modules/FindBoost.cmake:907 (_Boost_COMPONENT_DEPENDENCIES) /usr/share/cmake-3.10/Modules/FindBoost.cmake:1558 (_Boost_MISSING_DEPENDENCIES) cmake/Dependencies.cmake:8 (find_package) CMakeLists.txt:46 (include)
CMake Warning at /usr/share/cmake-3.10/Modules/FindBoost.cmake:801 (message): New Boost version may have incorrect or missing dependencies and imported targets Call Stack (most recent call first): /usr/share/cmake-3.10/Modules/FindBoost.cmake:907 (_Boost_COMPONENT_DEPENDENCIES) /usr/share/cmake-3.10/Modules/FindBoost.cmake:1558 (_Boost_MISSING_DEPENDENCIES) cmake/Dependencies.cmake:8 (find_package) CMakeLists.txt:46 (include)
CMake Warning at /usr/share/cmake-3.10/Modules/FindBoost.cmake:801 (message): New Boost version may have incorrect or missing dependencies and imported targets Call Stack (most recent call first): /usr/share/cmake-3.10/Modules/FindBoost.cmake:907 (_Boost_COMPONENT_DEPENDENCIES) /usr/share/cmake-3.10/Modules/FindBoost.cmake:1558 (_Boost_MISSING_DEPENDENCIES) cmake/Dependencies.cmake:8 (find_package) CMakeLists.txt:46 (include)
CMake Warning at /usr/share/cmake-3.10/Modules/FindBoost.cmake:801 (message): New Boost version may have incorrect or missing dependencies and imported targets Call Stack (most recent call first): /usr/share/cmake-3.10/Modules/FindBoost.cmake:907 (_Boost_COMPONENT_DEPENDENCIES) /usr/share/cmake-3.10/Modules/FindBoost.cmake:1558 (_Boost_MISSING_DEPENDENCIES) cmake/Dependencies.cmake:8 (find_package) CMakeLists.txt:46 (include)
CMake Warning at /usr/share/cmake-3.10/Modules/FindBoost.cmake:801 (message): New Boost version may have incorrect or missing dependencies and imported targets Call Stack (most recent call first): /usr/share/cmake-3.10/Modules/FindBoost.cmake:907 (_Boost_COMPONENT_DEPENDENCIES) /usr/share/cmake-3.10/Modules/FindBoost.cmake:1558 (_Boost_MISSING_DEPENDENCIES) cmake/Dependencies.cmake:8 (find_package) CMakeLists.txt:46 (include)
CMake Warning at /usr/share/cmake-3.10/Modules/FindBoost.cmake:801 (message): New Boost version may have incorrect or missing dependencies and imported targets Call Stack (most recent call first): /usr/share/cmake-3.10/Modules/FindBoost.cmake:907 (_Boost_COMPONENT_DEPENDENCIES) /usr/share/cmake-3.10/Modules/FindBoost.cmake:1558 (_Boost_MISSING_DEPENDENCIES) cmake/Dependencies.cmake:8 (find_package) CMakeLists.txt:46 (include)
-- Looking for pthread.h -- Looking for pthread.h - found -- Looking for pthread_create -- Looking for pthread_create - not found -- Looking for pthread_create in pthreads -- Looking for pthread_create in pthreads - not found -- Looking for pthread_create in pthread -- Looking for pthread_create in pthread - found -- Found Threads: TRUE
-- Boost version: 1.68.0 -- Found the following Boost libraries: -- system -- thread -- filesystem -- chrono -- date_time -- atomic -- Found GFlags: /usr/include
-- Found gflags (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libgflags.so) -- Found Glog: /usr/include
-- Found glog (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libglog.so) -- Found Protobuf: /usr/local/lib/libprotobuf.a;-lpthread (found version "3.5.0") -- Found PROTOBUF Compiler: /usr/local/bin/protoc -- HDF5: Using hdf5 compiler wrapper to determine C configuration -- HDF5: Using hdf5 compiler wrapper to determine CXX configuration -- Found HDF5: /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5_cpp.so;/usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so;/usr/lib/x86_64-linux-gnu/libpthread.so;/usr/lib/x86_64-linux-gnu/libsz.so;/usr/lib/x86_64-linux-gnu/libz.so;/usr/lib/x86_64-linux-gnu/libdl.so;/usr/lib/x86_64-linux-gnu/libm.so (found version "1.10.0.1") found components: HL -- Found LMDB: /usr/include
-- Found lmdb (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/liblmdb.so) -- Found LevelDB: /usr/include
-- Found LevelDB (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libleveldb.so) -- Found Snappy: /usr/include
-- Found Snappy (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libsnappy.so) -- CUDA detected: 10.0 -- Found cuDNN: ver. 7.3.1 found (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libcudnn.so) -- Added CUDA NVCC flags for: sm_61 -- OpenCV found (/usr/local/share/OpenCV) -- Found Atlas: /usr/include/x86_64-linux-gnu
-- Found Atlas (include: /usr/include/x86_64-linux-gnu library: /usr/lib/x86_64-linux-gnu/libatlas.so lapack: /usr/lib/x86_64-linux-gnu/liblapack.so -- Found PythonInterp: /usr/bin/python2.7 (found suitable version "2.7.15", minimum required is "2.7") -- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython2.7.so (found suitable version "2.7.15rc1", minimum required is "2.7") -- Found NumPy: /usr/local/lib/python2.7/dist-packages/numpy/core/include (found suitable version "1.15.4", minimum required is "1.7.1") -- NumPy ver. 1.15.4 found (include: /usr/local/lib/python2.7/dist-packages/numpy/core/include) CMake Warning at /usr/share/cmake-3.10/Modules/FindBoost.cmake:801 (message): New Boost version may have incorrect or missing dependencies and imported targets Call Stack (most recent call first): /usr/share/cmake-3.10/Modules/FindBoost.cmake:907 (_Boost_COMPONENT_DEPENDENCIES) /usr/share/cmake-3.10/Modules/FindBoost.cmake:1558 (_Boost_MISSING_DEPENDENCIES) cmake/Dependencies.cmake:172 (find_package) CMakeLists.txt:46 (include)
-- Could NOT find Boost -- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) -- Python interface is disabled or not all required dependencies found. Building without it... -- Found Git: /usr/bin/git (found version "2.17.1") -- -- Caffe Configuration Summary -- General: -- Version : 1.0.0 -- Git : unknown -- System : Linux -- C++ compiler : /usr/bin/c++ -- Release CXX flags : -O3 -DNDEBUG -fPIC -Wall -Wno-sign-compare -Wno-uninitialized -- Debug CXX flags : -g -fPIC -Wall -Wno-sign-compare -Wno-uninitialized -- Build type : Release -- -- BUILD_SHARED_LIBS : ON -- BUILD_python : ON -- BUILD_matlab : OFF -- BUILD_docs : ON -- CPU_ONLY : OFF -- USE_OPENCV : ON -- USE_LEVELDB : ON -- USE_LMDB : ON -- USE_NCCL : OFF -- ALLOW_LMDB_NOLOCK : OFF -- -- Dependencies: -- BLAS : Yes (Atlas) -- Boost : Yes (ver. 1.68) -- glog : Yes -- gflags : Yes -- protobuf : Yes (ver. 3.5.0) -- lmdb : Yes (ver. 0.9.21) -- LevelDB : Yes (ver. 1.20) -- Snappy : Yes (ver. ..) -- OpenCV : Yes (ver. 3.4.0) -- CUDA : Yes (ver. 10.0) -- -- NVIDIA CUDA: -- Target GPU(s) : Auto -- GPU arch(s) : sm_61 -- cuDNN : Yes (ver. 7.3.1) -- -- Documentaion: -- Doxygen : No -- config_file :
-- -- Install: -- Install path : /media/wjl/New/caffese/caffe-4d2400e7ae692b25f034f02ff8e8cd3621725f5c/build/install -- CMake Error: The following variables are used in this project, but they are set to NOTFOUND. Please set them or make sure they are set and tested correctly in the CMake files: CUDA_cublas_device_LIBRARY (ADVANCED) linked by target "caffe" in directory /media/wjl/New/caffese/caffe-4d2400e7ae692b25f034f02ff8e8cd3621725f5c/src/caffe
-- Configuring incomplete, errors occurred! See also "/media/wjl/New/caffese/caffe-4d2400e7ae692b25f034f02ff8e8cd3621725f5c/build/CMakeFiles/CMakeOutput.log". See also "/media/wjl/New/caffese/caffe-4d2400e7ae692b25f034f02ff8e8cd3621725f5c/build/CMakeFiles/CMakeError.log".