search

andersbll / cudarray

CUDA-based NumPy
MIT License
234 stars 61 forks source link

setup.py install problem #22

Closed rmurray2 closed 9 years ago

rmurray2 commented 9 years ago

Greetings, great project here; trying to get this to work with your fantastic neural artistic style project so the generation of images there doesn't take days (on a CPU).

As I don't have a GPU, I set up an Amazon GPU instance provided by Nvidia called, "Amazon Linux AMI with NVIDIA GRID GPU Driver", and downloaded the cudnn package from Nvidia's website.

Now when I try to install your package I get the following error. I pasted all the way from the make command:

[ec2-user@ip-172-31-3-36 cudarray-master]$ make g++ -DCUDNN_ENABLED -O3 -fPIC -Wall -Wfatal-errors -I./include -I/opt/nvidia/cuda/include -c -o src/nnet/conv_bc01_matmul.o src/nnet/conv_bc01_matmul.cpp nvcc -gencode arch=compute_20,code=sm_20 -gencode arch=compute_20,code=compute_20 -gencode arch=compute_30,code=sm_30 -gencode arch=compute_30,code=compute_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_35,code=compute_35 -O3 --compiler-options '-DCUDNN_ENABLED -O3 -fPIC -Wall -Wfatal-errors' --ftz=true --prec-div=false -prec-sqrt=false --fmad=true -I./include -I/opt/nvidia/cuda/include -c -o src/nnet/pool_b01.o src/nnet/pool_b01.cu g++ -DCUDNN_ENABLED -O3 -fPIC -Wall -Wfatal-errors -I./include -I/opt/nvidia/cuda/include -c -o src/nnet/cudnn.o src/nnet/cudnn.cpp nvcc -gencode arch=compute_20,code=sm_20 -gencode arch=compute_20,code=compute_20 -gencode arch=compute_30,code=sm_30 -gencode arch=compute_30,code=compute_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_35,code=compute_35 -O3 --compiler-options '-DCUDNN_ENABLED -O3 -fPIC -Wall -Wfatal-errors' --ftz=true --prec-div=false -prec-sqrt=false --fmad=true -I./include -I/opt/nvidia/cuda/include -c -o src/array_ops.o src/array_ops.cu nvcc -gencode arch=compute_20,code=sm_20 -gencode arch=compute_20,code=compute_20 -gencode arch=compute_30,code=sm_30 -gencode arch=compute_30,code=compute_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_35,code=compute_35 -O3 --compiler-options '-DCUDNN_ENABLED -O3 -fPIC -Wall -Wfatal-errors' --ftz=true --prec-div=false -prec-sqrt=false --fmad=true -I./include -I/opt/nvidia/cuda/include -c -o src/elementwise.o src/elementwise.cu nvcc -gencode arch=compute_20,code=sm_20 -gencode arch=compute_20,code=compute_20 -gencode arch=compute_30,code=sm_30 -gencode arch=compute_30,code=compute_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_35,code=compute_35 -O3 --compiler-options '-DCUDNN_ENABLED -O3 -fPIC -Wall -Wfatal-errors' --ftz=true --prec-div=false -prec-sqrt=false --fmad=true -I./include -I/opt/nvidia/cuda/include -c -o src/reduction.o src/reduction.cu g++ -DCUDNN_ENABLED -O3 -fPIC -Wall -Wfatal-errors -I./include -I/opt/nvidia/cuda/include -c -o src/blas.o src/blas.cpp nvcc -gencode arch=compute_20,code=sm_20 -gencode arch=compute_20,code=compute_20 -gencode arch=compute_30,code=sm_30 -gencode arch=compute_30,code=compute_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_35,code=compute_35 -O3 --compiler-options '-DCUDNN_ENABLED -O3 -fPIC -Wall -Wfatal-errors' --ftz=true --prec-div=false -prec-sqrt=false --fmad=true -I./include -I/opt/nvidia/cuda/include -c -o src/random.o src/random.cu nvcc -gencode arch=compute_20,code=sm_20 -gencode arch=compute_20,code=compute_20 -gencode arch=compute_30,code=sm_30 -gencode arch=compute_30,code=compute_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_35,code=compute_35 -O3 --compiler-options '-DCUDNN_ENABLED -O3 -fPIC -Wall -Wfatal-errors' --ftz=true --prec-div=false -prec-sqrt=false --fmad=true -I./include -I/opt/nvidia/cuda/include -c -o src/image/img2win.o src/image/img2win.cu nvcc -gencode arch=compute_20,code=sm_20 -gencode arch=compute_20,code=compute_20 -gencode arch=compute_30,code=sm_30 -gencode arch=compute_30,code=compute_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_35,code=compute_35 -O3 --compiler-options '-DCUDNN_ENABLED -O3 -fPIC -Wall -Wfatal-errors' --ftz=true --prec-div=false -prec-sqrt=false --fmad=true -I./include -I/opt/nvidia/cuda/include -c -o src/nnet/one_hot.o src/nnet/one_hot.cu mkdir -p ./build g++ -shared -DCUDNN_ENABLED -O3 -fPIC -Wall -Wfatal-errors -o build/libcudarray.so src/nnet/conv_bc01_matmul.o src/nnet/pool_b01.o src/nnet/cudnn.o src/array_ops.o src/elementwise.o src/reduction.o src/blas.o src/random.o src/image/img2win.o src/nnet/one_hot.o -L/opt/nvidia/cuda/lib64 -L/opt/nvidia/cuda/lib -lcudart -lcublas -lcufft -lcurand -lcudnn [ec2-user@ip-172-31-3-36 cudarray-master]$ sudo make install cp ./build/libcudarray.so /usr/local/lib/libcudarray.so [ec2-user@ip-172-31-3-36 cudarray-master]$ sudo python setup.py install Compiling cudarray/numpy_backend/nnet/conv_bc01.pyx because it changed. Compiling cudarray/numpy_backend/nnet/pool_bc01.pyx because it changed. Compiling cudarray/numpy_backend/nnet/lrnorm_bc01.pyx because it changed. [1/3] Cythonizing cudarray/numpy_backend/nnet/conv_bc01.pyx [2/3] Cythonizing cudarray/numpy_backend/nnet/lrnorm_bc01.pyx [3/3] Cythonizing cudarray/numpy_backend/nnet/pool_bc01.pyx /usr/lib/python2.7/dist-packages/setuptools/dist.py:282: UserWarning: Normalizing '0.1.dev' to '0.1.dev0' normalized_version, running install running bdist_egg running egg_info creating cudarray.egg-info writing requirements to cudarray.egg-info/requires.txt writing cudarray.egg-info/PKG-INFO writing top-level names to cudarray.egg-info/top_level.txt writing dependency_links to cudarray.egg-info/dependency_links.txt writing manifest file 'cudarray.egg-info/SOURCES.txt' reading manifest file 'cudarray.egg-info/SOURCES.txt' writing manifest file 'cudarray.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py creating build/lib.linux-x86_64-2.7 creating build/lib.linux-x86_64-2.7/cudarray copying cudarray/elementwise.py -> build/lib.linux-x86_64-2.7/cudarray copying cudarray/reduction.py -> build/lib.linux-x86_64-2.7/cudarray copying cudarray/base.py -> build/lib.linux-x86_64-2.7/cudarray copying cudarray/random.py -> build/lib.linux-x86_64-2.7/cudarray copying cudarray/cudarray.py -> build/lib.linux-x86_64-2.7/cudarray copying cudarray/helpers.py -> build/lib.linux-x86_64-2.7/cudarray copying cudarray/linalg.py -> build/lib.linux-x86_64-2.7/cudarray copying cudarray/init.py -> build/lib.linux-x86_64-2.7/cudarray creating build/lib.linux-x86_64-2.7/cudarray/wrap copying cudarray/wrap/init.py -> build/lib.linux-x86_64-2.7/cudarray/wrap creating build/lib.linux-x86_64-2.7/cudarray/numpy_backend copying cudarray/numpy_backend/init.py -> build/lib.linux-x86_64-2.7/cudarray/numpy_backend creating build/lib.linux-x86_64-2.7/cudarray/nnet copying cudarray/nnet/special.py -> build/lib.linux-x86_64-2.7/cudarray/nnet copying cudarray/nnet/math.py -> build/lib.linux-x86_64-2.7/cudarray/nnet copying cudarray/nnet/pool.py -> build/lib.linux-x86_64-2.7/cudarray/nnet copying cudarray/nnet/init.py -> build/lib.linux-x86_64-2.7/cudarray/nnet copying cudarray/nnet/conv.py -> build/lib.linux-x86_64-2.7/cudarray/nnet creating build/lib.linux-x86_64-2.7/cudarray/batch copying cudarray/batch/linalg.py -> build/lib.linux-x86_64-2.7/cudarray/batch copying cudarray/batch/init.py -> build/lib.linux-x86_64-2.7/cudarray/batch creating build/lib.linux-x86_64-2.7/cudarray/numpy_backend/nnet copying cudarray/numpy_backend/nnet/special.py -> build/lib.linux-x86_64-2.7/cudarray/numpy_backend/nnet copying cudarray/numpy_backend/nnet/activations.py -> build/lib.linux-x86_64-2.7/cudarray/numpy_backend/nnet copying cudarray/numpy_backend/nnet/pool.py -> build/lib.linux-x86_64-2.7/cudarray/numpy_backend/nnet copying cudarray/numpy_backend/nnet/init.py -> build/lib.linux-x86_64-2.7/cudarray/numpy_backend/nnet copying cudarray/numpy_backend/nnet/conv.py -> build/lib.linux-x86_64-2.7/cudarray/numpy_backend/nnet running build_ext building 'cudarray.numpy_backend.nnet.conv_bc01' extension creating build/temp.linux-x86_64-2.7 creating build/temp.linux-x86_64-2.7/cudarray creating build/temp.linux-x86_64-2.7/cudarray/numpy_backend creating build/temp.linux-x86_64-2.7/cudarray/numpy_backend/nnet gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/local/lib64/python2.7/site-packages/numpy/core/include -I/usr/include/python2.7 -c cudarray/numpy_backend/nnet/conv_bc01.c -o build/temp.linux-x86_64-2.7/cudarray/numpy_backend/nnet/conv_bc01.o In file included from /usr/local/lib64/python2.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1804:0, from /usr/local/lib64/python2.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:17, from /usr/local/lib64/python2.7/site-packages/numpy/core/include/numpy/arrayobject.h:4, from cudarray/numpy_backend/nnet/conv_bc01.c:250: /usr/local/lib64/python2.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]

warning "Using deprecated NumPy API, disable it by " \

^ In file included from /usr/local/lib64/python2.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:26:0, from /usr/local/lib64/python2.7/site-packages/numpy/core/include/numpy/arrayobject.h:4, from cudarray/numpy_backend/nnet/conv_bc01.c:250: /usr/local/lib64/python2.7/site-packages/numpy/core/include/numpy/__multiarray_api.h:1629:1: warning: ‘_import_array’ defined but not used [-Wunused-function] _import_array(void) ^ In file included from /usr/local/lib64/python2.7/site-packages/numpy/core/include/numpy/ufuncobject.h:317:0, from cudarray/numpy_backend/nnet/conv_bc01.c:251: /usr/local/lib64/python2.7/site-packages/numpy/core/include/numpy/__ufunc_api.h:241:1: warning: ‘_import_umath’ defined but not used [-Wunused-function] _import_umath(void) ^ gcc -pthread -shared build/temp.linux-x86_64-2.7/cudarray/numpy_backend/nnet/conv_bc01.o -L/usr/lib64 -lpython2.7 -o build/lib.linux-x86_64-2.7/cudarray/numpy_backend/nnet/conv_bc01.so building 'cudarray.numpy_backend.nnet.pool_bc01' extension gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/local/lib64/python2.7/site-packages/numpy/core/include -I/usr/include/python2.7 -c cudarray/numpy_backend/nnet/pool_bc01.c -o build/temp.linux-x86_64-2.7/cudarray/numpy_backend/nnet/pool_bc01.o In file included from /usr/local/lib64/python2.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1804:0, from /usr/local/lib64/python2.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:17, from /usr/local/lib64/python2.7/site-packages/numpy/core/include/numpy/arrayobject.h:4, from cudarray/numpy_backend/nnet/pool_bc01.c:250: /usr/local/lib64/python2.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]

warning "Using deprecated NumPy API, disable it by " \

^ In file included from /usr/local/lib64/python2.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:26:0, from /usr/local/lib64/python2.7/site-packages/numpy/core/include/numpy/arrayobject.h:4, from cudarray/numpy_backend/nnet/pool_bc01.c:250: /usr/local/lib64/python2.7/site-packages/numpy/core/include/numpy/__multiarray_api.h:1629:1: warning: ‘_import_array’ defined but not used [-Wunused-function] _import_array(void) ^ In file included from /usr/local/lib64/python2.7/site-packages/numpy/core/include/numpy/ufuncobject.h:317:0, from cudarray/numpy_backend/nnet/pool_bc01.c:251: /usr/local/lib64/python2.7/site-packages/numpy/core/include/numpy/__ufunc_api.h:241:1: warning: ‘_import_umath’ defined but not used [-Wunused-function] _import_umath(void) ^ gcc -pthread -shared build/temp.linux-x86_64-2.7/cudarray/numpy_backend/nnet/pool_bc01.o -L/usr/lib64 -lpython2.7 -o build/lib.linux-x86_64-2.7/cudarray/numpy_backend/nnet/pool_bc01.so building 'cudarray.numpy_backend.nnet.lrnorm_bc01' extension gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/local/lib64/python2.7/site-packages/numpy/core/include -I/usr/include/python2.7 -c cudarray/numpy_backend/nnet/lrnorm_bc01.c -o build/temp.linux-x86_64-2.7/cudarray/numpy_backend/nnet/lrnorm_bc01.o In file included from /usr/local/lib64/python2.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1804:0, from /usr/local/lib64/python2.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:17, from /usr/local/lib64/python2.7/site-packages/numpy/core/include/numpy/arrayobject.h:4, from cudarray/numpy_backend/nnet/lrnorm_bc01.c:250: /usr/local/lib64/python2.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]

warning "Using deprecated NumPy API, disable it by " \

^ In file included from /usr/local/lib64/python2.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:26:0, from /usr/local/lib64/python2.7/site-packages/numpy/core/include/numpy/arrayobject.h:4, from cudarray/numpy_backend/nnet/lrnorm_bc01.c:250: /usr/local/lib64/python2.7/site-packages/numpy/core/include/numpy/__multiarray_api.h:1629:1: warning: ‘_import_array’ defined but not used [-Wunused-function] _import_array(void) ^ In file included from /usr/local/lib64/python2.7/site-packages/numpy/core/include/numpy/ufuncobject.h:317:0, from cudarray/numpy_backend/nnet/lrnorm_bc01.c:251: /usr/local/lib64/python2.7/site-packages/numpy/core/include/numpy/__ufunc_api.h:241:1: warning: ‘_import_umath’ defined but not used [-Wunused-function] _import_umath(void) ^ gcc -pthread -shared build/temp.linux-x86_64-2.7/cudarray/numpy_backend/nnet/lrnorm_bc01.o -L/usr/lib64 -lpython2.7 -o build/lib.linux-x86_64-2.7/cudarray/numpy_backend/nnet/lrnorm_bc01.so cythoning ./cudarray/wrap/cudart.pyx to ./cudarray/wrap/cudart.cpp building 'cudarray.wrap.cudart' extension creating build/temp.linux-x86_64-2.7/cudarray/wrap gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/local/cuda/include -I./include -I/usr/local/lib64/python2.7/site-packages/numpy/core/include -I/usr/local/lib64/python2.7/site-packages/numpy/core/include -I/usr/include/python2.7 -c ./cudarray/wrap/cudart.cpp -o build/temp.linux-x86_64-2.7/./cudarray/wrap/cudart.o -O3 -fPIC -Wall -Wfatal-errors ./cudarray/wrap/cudart.cpp:248:26: fatal error: driver_types.h: No such file or directory

include "driver_types.h"

                      ^

compilation terminated. error: command 'gcc' failed with exit status 1

Any help would be greatly appreciated; am excited to get the artistic style program working at a reasonable speed.

andersbll commented 9 years ago

driver_types.h is part of the CUDA SDK. Make sure it is installed and that your CUDA_PREFIX points to the right location. Read more here.

rmurray2 commented 9 years ago

Thanks for your prompt reply.

When I do ldconfig -p | grep libcuda one of the results is /opt/nvidia/cuda/. This dir contains the bin/, lib/, include/ folders, so the CUDA_PREFIX env var was set to that value. Additionally, when I try to install the CUDA SDK, I get a notification that the package is already installed, so I assume that the package is installed and that the env var points to the right place.

Furthermore, sudo find / -name driver_types.h shows a result in /opt/nvidia/cuda/include/.

andersbll commented 9 years ago

Ah! Are you doing sudo python setup.py install? In that case you need to transfer environment variables using sudo -E python setup.py install.