search

BVLC / caffe

Caffe: a fast open framework for deep learning.
http://caffe.berkeleyvision.org/
Other
34.01k stars 18.71k forks source link

Make runtest failed on Ubuntu 16.04 and CUDA 10.0 #6861

Open earlysleepearlyup opened 4 years ago

earlysleepearlyup commented 4 years ago

Issue summary

Make runtest failed on Ubuntu 16.04 and CUDA 10.0

[----------] 6 tests from CuDNNConvolutionLayerTest/1, where TypeParam = double [ RUN ] CuDNNConvolutionLayerTest/1.TestGradientCuDNN Aborted at 1573197090 (unix time) try "date -d @1573197090" if you are using GNU date PC: @ 0x7f33bb11fdab caffe::SyncedMemory::cpu_data() SIGSEGV (@0x30000001b) received by PID 6871 (TID 0x7f33c646e740) from PID 27; stack trace: @ 0x7f33ba898390 (unknown) @ 0x7f33bb11fdab caffe::SyncedMemory::cpu_data() @ 0x7f33bb128f12 caffe::Blob<>::cpu_data() @ 0x7f33bb180d4b caffe::CuDNNConvolutionLayer<>::LayerSetUp() @ 0x4626fe caffe::GradientChecker<>::CheckGradientExhaustive() @ 0x7314d6 caffe::CuDNNConvolutionLayerTest_TestGradientCuDNN_Test<>::TestBody() @ 0x7de513 testing::internal::HandleExceptionsInMethodIfSupported<>() @ 0x7d7b2a testing::Test::Run() @ 0x7d7c78 testing::TestInfo::Run() @ 0x33d47ee0 (unknown) Makefile:477: recipe for target 'runtest' failed make: *** [runtest] Segmentation fault

Steps to reproduce

make clean make all -j16 make test -j16 make runtest

Tried solutions

modify the makefile.config

## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!

# cuDNN acceleration switch (uncomment to build with cuDNN).
USE_CUDNN := 1

# CPU-only switch (uncomment to build without GPU support).
# CPU_ONLY := 1

# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++

# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda-10.0
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr

# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 lines for compatibility.
CUDA_ARCH := -gencode arch=compute_30,code=sm_30 \
        -gencode arch=compute_35,code=sm_35 \
        -gencode arch=compute_50,code=sm_50 \
        -gencode arch=compute_52,code=sm_52 \
            -gencode arch=compute_60,code=sm_60 \
            -gencode arch=compute_61,code=sm_61 \
            -gencode arch=compute_61,code=compute_61 

# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := atlas
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
# BLAS_INCLUDE := /path/to/your/blas
# BLAS_LIB := /path/to/your/blas

# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib

# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
MATLAB_DIR := /usr/local/MATLAB/R2018b
# MATLAB_DIR := /Applications/MATLAB_R2012b.app

# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
PYTHON_INCLUDE := /usr/include/python3.5 \
        /home/liying/.local/lib/python3.5/site-packages/numpy/core/include/numpy    
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
# ANACONDA_HOME := $(HOME)/anaconda
# PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
        # $(ANACONDA_HOME)/include/python2.7 \
        # $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \

# We need to be able to find libpythonX.X.so or .dylib.
PYTHON_LIB := /usr/lib
# PYTHON_LIB := $(ANACONDA_HOME)/lib

# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib

# Uncomment to support layers written in Python (will link against Python libs)
# WITH_PYTHON_LAYER := 1

# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial /usr/local/include/opencv /usr/local/include/opencv2
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/hdf5/serial

# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib

# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
USE_PKG_CONFIG := 1

BUILD_DIR := build
DISTRIBUTE_DIR := distribute

# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1

# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0

# enable pretty build (comment to see full commands)
Q ?= @

System configuration

kikiyami commented 4 years ago 
I had tried the version and worked fine as below:
Operating system: Ubuntu 16.04
Compiler: GNU make 4.1
CUDA version (if applicable): CUDA 10.0
CUDNN version (if applicable): CUDNN 7.3.1.20
BLAS: open
Python version (if using pycaffe): python 3.5
MATLAB version (if using matcaffe): N/A