search

NVIDIA / MinkowskiEngine

Minkowski Engine is an auto-diff neural network library for high-dimensional sparse tensors
https://nvidia.github.io/MinkowskiEngine
Other
2.47k stars 367 forks source link

Compilation issue. #12

Closed XuyangBai closed 5 years ago

XuyangBai commented 5 years ago

Hello, thank you for your sharing. I followed the instruction but still got the following error src/common.hpp:40:10: fatal error: cublas_v2.h: No such file or directory . I have tried update the CUDA_DIR in Makefile to /usr/local/cuda but it still shows the same error message.

chrischoy commented 5 years ago

Hmm. It is difficult to see what is happening exactly. Could you print the output of

python -c 'from torch.utils.cpp_extension import _find_cuda_home; print(_find_cuda_home())'

python -c 'from torch.utils.cpp_extension import include_paths; [print(p) for p in include_paths()]'

/usr/local/cuda/bin/nvcc -V

ls /usr/local/cuda/include

locate cublas_v2.h
XuyangBai commented 5 years ago

:/usr/local/cuda

/home/xybai/anaconda3/envs/py3-mink/lib/python3.7/site-packages/torch/include
/home/xybai/anaconda3/envs/py3-mink/lib/python3.7/site-packages/torch/include/torch/csrc/api/include
/home/xybai/anaconda3/envs/py3-mink/lib/python3.7/site-packages/torch/include/TH
/home/xybai/anaconda3/envs/py3-mink/lib/python3.7/site-packages/torch/include/THC
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2017 NVIDIA Corporation
Built on Fri_Sep__1_21:08:03_CDT_2017
Cuda compilation tools, release 9.0, V9.0.176

For ls /usr/local/cuda/include I can see cublas_v2.h

/usr/local/cuda-9.0/include/cublas_v2.h

chrischoy commented 5 years ago

Hmm. It seems that everything is there.

Could you comment line 5 in the Makefile and build again by simply typing make and put the result here?

I want to make sure that the g++ gets all the INCLUD_DIRS right.

XuyangBai commented 5 years ago

NVCC src/pooling_max.cu nvcc -std=c++11 -ccbin=g++ -Xcompiler -fPIC -DNDEBUG -O3 -I./ -I:/usr/local/cuda/include -I/home/xybai/anaconda3/envs/py3-mink/include/python3.7m -I/home/xybai/anaconda3/envs/py3-mink/include/python3.7m/.. -I/home/xybai/anaconda3/envs/py3-mink/lib/python3.7/site-packages/torch/include -I/home/xybai/anaconda3/envs/py3-mink/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/xybai/anaconda3/envs/py3-mink/lib/python3.7/site-packages/torch/include/TH -I/home/xybai/anaconda3/envs/py3-mink/lib/python3.7/site-packages/torch/include/THC -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=minkowski -D_GLIBCXX_USE_CXX11_ABI=0 -gencode arch=compute_30,code=sm_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_50,code=sm_50 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_61,code=compute_61 -M src/pooling_max.cu -o objs/cuda/pooling_max.d \ -odir objs/cuda In file included from /usr/local/cuda/bin/..//include/host_config.h:50:0, from /usr/local/cuda/bin/..//include/cuda_runtime.h:78, from :0: /usr/local/cuda/bin/..//include/crt/host_config.h:119:2: error: #error -- unsupported GNU version! gcc versions later than 6 are not supported!

error -- unsupported GNU version! gcc versions later than 6 are not supported!

^~~~~ Makefile:124: recipe for target 'objs/cuda/pooling_max.o' failed make: *** [objs/cuda/pooling_max.o] Error 1

chrischoy commented 5 years ago

I think this is a different error.

 #error -- unsupported GNU version! gcc versions later than 6 are not supported!

Could you install g++ > v6.0?

sudo apt-get update && \
sudo apt-get install build-essential software-properties-common -y && \
sudo apt-get install gcc-6 g++-6 -y && \
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-6 60 --slave /usr/bin/g++ g++ /usr/bin/g++-6 && \
gcc -v
XuyangBai commented 5 years ago

Hi, Thank you so much for your help. I have run 

sudo apt-get update && \
sudo apt-get install build-essential software-properties-common -y && \
sudo apt-get install gcc-6 g++-6 -y && \

And change the 8 line of Makefile to CXX := g++-6, and run make. But still got the following error

CXX src/pooling_global.cpp
g++-6 src/pooling_global.cpp -MMD -MP -pthread -fPIC -fwrapv -std=c++11 -DNDEBUG -O3 -I./ -I:/usr/local/cuda/include -I/home/xybai/anaconda3/envs/py3-mink/include/python3.7m -I/home/xybai/anaconda3/envs/py3-mink/include/python3.7m/.. -I/home/xybai/anaconda3/envs/py3-mink/lib/python3.7/site-packages/torch/include -I/home/xybai/anaconda3/envs/py3-mink/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/xybai/anaconda3/envs/py3-mink/lib/python3.7/site-packages/torch/include/TH -I/home/xybai/anaconda3/envs/py3-mink/lib/python3.7/site-packages/torch/include/THC -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=minkowski -D_GLIBCXX_USE_CXX11_ABI=0 -Wall -Wno-sign-compare -Wcomment -c -o objs/pooling_global.o
In file included from src/pooling_global.cpp:25:0:
src/common.hpp:40:23: fatal error: cublas_v2.h: No such file or directory
 #include <cublas_v2.h>
                       ^
compilation terminated.
Makefile:120: recipe for target 'objs/pooling_global.o' failed
make: *** [objs/pooling_global.o] Error 1

The output of gcc-6 -v is 

Using built-in specs.
COLLECT_GCC=gcc-6
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/6/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 6.5.0-2ubuntu1~18.04' --with-bugurl=file:///usr/share/doc/gcc-6/README.Bugs --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++ --prefix=/usr --with-as=/usr/bin/x86_64-linux-gnu-as --with-ld=/usr/bin/x86_64-linux-gnu-ld --program-suffix=-6 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 6.5.0 20181026 (Ubuntu 6.5.0-2ubuntu1~18.04)

So I think g++-6 is already installed. Do I need to change somewhere else to make it use g++-6 to compile?

chrischoy commented 5 years ago

There is a typo.

-I:/usr/local/cuda/include

This should be -I/usr/local/cuda/include

Please remove the semicolon or reset the Makefile to default.

XuyangBai commented 5 years ago

Thank you so much for your help !

cgage2 commented 5 years ago

Hi, I am dealing with a somewhat similar issue, everything works/compiles as expected until it hits this gcc call. The problem seems to be a misplaced colon as above (when I copy paste the command and remove the colon it compiles) but I cant find reference to this command anywhere in the Makefile and changing the CUDA_DIR to has no effect. Thanks for your time and sharing your code.

rm -f -rf build dist
running install
running bdist_egg
running egg_info
writing MinkowskiEngine.egg-info/PKG-INFO
writing dependency_links to MinkowskiEngine.egg-info/dependency_links.txt
writing requirements to MinkowskiEngine.egg-info/requires.txt
writing top-level names to MinkowskiEngine.egg-info/top_level.txt
reading manifest file 'MinkowskiEngine.egg-info/SOURCES.txt'
writing manifest file 'MinkowskiEngine.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib.linux-x86_64-3.7
creating build/lib.linux-x86_64-3.7/MinkowskiEngine
copying ./MinkowskiEngine/MinkowskiNormalization.py -> build/lib.linux-x86_64-3.7/MinkowskiEngine
copying ./MinkowskiEngine/MinkowskiPruning.py -> build/lib.linux-x86_64-3.7/MinkowskiEngine
copying ./MinkowskiEngine/MinkowskiPooling.py -> build/lib.linux-x86_64-3.7/MinkowskiEngine
copying ./MinkowskiEngine/__init__.py -> build/lib.linux-x86_64-3.7/MinkowskiEngine
copying ./MinkowskiEngine/Common.py -> build/lib.linux-x86_64-3.7/MinkowskiEngine
copying ./MinkowskiEngine/SparseTensor.py -> build/lib.linux-x86_64-3.7/MinkowskiEngine
copying ./MinkowskiEngine/MinkowskiOps.py -> build/lib.linux-x86_64-3.7/MinkowskiEngine
copying ./MinkowskiEngine/MinkowskiBroadcast.py -> build/lib.linux-x86_64-3.7/MinkowskiEngine
copying ./MinkowskiEngine/MinkowskiNonlinearity.py -> build/lib.linux-x86_64-3.7/MinkowskiEngine
copying ./MinkowskiEngine/MinkowskiFunctional.py -> build/lib.linux-x86_64-3.7/MinkowskiEngine
copying ./MinkowskiEngine/MinkowskiNetwork.py -> build/lib.linux-x86_64-3.7/MinkowskiEngine
copying ./MinkowskiEngine/MinkowskiCoords.py -> build/lib.linux-x86_64-3.7/MinkowskiEngine
copying ./MinkowskiEngine/MinkowskiConvolution.py -> build/lib.linux-x86_64-3.7/MinkowskiEngine
creating build/lib.linux-x86_64-3.7/MinkowskiEngine/utils
copying ./MinkowskiEngine/utils/collation.py -> build/lib.linux-x86_64-3.7/MinkowskiEngine/utils
copying ./MinkowskiEngine/utils/__init__.py -> build/lib.linux-x86_64-3.7/MinkowskiEngine/utils
copying ./MinkowskiEngine/utils/gradcheck.py -> build/lib.linux-x86_64-3.7/MinkowskiEngine/utils
copying ./MinkowskiEngine/utils/voxelization.py -> build/lib.linux-x86_64-3.7/MinkowskiEngine/utils
creating build/lib.linux-x86_64-3.7/MinkowskiEngine/modules
copying ./MinkowskiEngine/modules/__init__.py -> build/lib.linux-x86_64-3.7/MinkowskiEngine/modules
copying ./MinkowskiEngine/modules/resnet_block.py -> build/lib.linux-x86_64-3.7/MinkowskiEngine/modules
copying ./MinkowskiEngine/modules/senet_block.py -> build/lib.linux-x86_64-3.7/MinkowskiEngine/modules
running build_ext
building 'MinkowskiEngineBackend' extension
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/pybind
gcc -pthread -B /home/ubuntu/.conda/envs/mink/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I./ -I/home/ubuntu/.conda/envs/mink/include/python3.7m/.. -I/home/ubuntu/.conda/envs/mink/lib/python3.7/site-packages/torch/include -I/home/ubuntu/.conda/envs/mink/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/ubuntu/.conda/envs/mink/lib/python3.7/site-packages/torch/include/TH -I/home/ubuntu/.conda/envs/mink/lib/python3.7/site-packages/torch/include/THC -I:/usr/local/cuda:/usr/local/cuda-10.0/include -I/home/ubuntu/.conda/envs/mink/include/python3.7m -c pybind/minkowski.cpp -o build/temp.linux-x86_64-3.7/pybind/minkowski.o -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=MinkowskiEngineBackend -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
In file included from pybind/extern.hpp:28:0,
                 from pybind/minkowski.cpp:32:
./src/common.hpp:40:23: fatal error: cublas_v2.h: No such file or directory
compilation terminated.
chrischoy commented 5 years ago

It says

./src/common.hpp:40:23: fatal error: cublas_v2.h: No such file or directory

Could you print out

python -c 'from torch.utils.cpp_extension import _find_cuda_home; print(_find_cuda_home())'

python -c 'from torch.utils.cpp_extension import include_paths; [print(p) for p in include_paths()]'

/usr/local/cuda/bin/nvcc -V

ls /usr/local/cuda/include

locate cublas_v2.h

The first line python -c 'from torch.utils.cpp_extension import _find_cuda_home; print(_find_cuda_home())' should return something like /usr/local/cuda

cgage2 commented 5 years ago

The first line returns :/usr/local/cuda:/usr/local/cuda-10.0 I previously replaced it in the Makefile with both CUDA_DIR := /usr/local/cuda and CUDA_DIR := /usr/local/cuda-10.0 and the libraries all compiled up until my error above.

/home/ubuntu/.conda/envs/mink/lib/python3.7/site-packages/torch/include
/home/ubuntu/.conda/envs/mink/lib/python3.7/site-packages/torch/include/torch/csrc/api/include
/home/ubuntu/.conda/envs/mink/lib/python3.7/site-packages/torch/include/TH
/home/ubuntu/.conda/envs/mink/lib/python3.7/site-packages/torch/include/THC
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2018 NVIDIA Corporation
Built on Sat_Aug_25_21:08:01_CDT_2018
Cuda compilation tools, release 10.0, V10.0.130

ls /usr/local/cuda/include -> cublas_v2.h is present in the list

/usr/local/cuda-10.0/targets/x86_64-linux/include/cublas_v2.h

chrischoy commented 5 years ago

Okay, I think the error is in the CPPExtension

gcc -pthread -B /home/ubuntu/.conda/envs/mink/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I./ -I/home/ubuntu/.conda/envs/mink/include/python3.7m/.. -I/home/ubuntu/.conda/envs/mink/lib/python3.7/site-packages/torch/include -I/home/ubuntu/.conda/envs/mink/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/ubuntu/.conda/envs/mink/lib/python3.7/site-packages/torch/include/TH -I/home/ubuntu/.conda/envs/mink/lib/python3.7/site-packages/torch/include/THC -I:/usr/local/cuda:/usr/local/cuda-10.0/include -I/home/ubuntu/.conda/envs/mink/include/python3.7m -c pybind/minkowski.cpp -o build/temp.linux-x86_64-3.7/pybind/minkowski.o -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=MinkowskiEngineBackend -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11

Here, it is using -I:/usr/local/cuda:/usr/local/cuda-10.0/include which should have been -I/usr/local/cuda.

This command is generated from pytorch and it assumes that _find_cuda_home function generates a valid path (from torch.utils.cpp_extension import _find_cuda_home).

Did you manually changed the CUDA_HOME directory? If so, could you reset back to export CUDA_HOME=/usr/local/cuda?

cgage2 commented 5 years ago

I don't remember changing it but its in my .bashrc as export CUDA_HOME=${CUDA_HOME}:/usr/local/cuda:/usr/local/cuda-10.0 I changed my CUDA_HOME as you said and it installed successfully. Thanks for your help. (ps: Turns out we have a packer script automatically adding it to .bashrc.)

chrischoy commented 5 years ago

Thanks for reporting the issue. I updated the common issue page since it happens frequently! https://stanfordvl.github.io/MinkowskiEngine/issues.html