Build Caffe with clBlas but it requires ViennaCL

[sambookhon]

Hi @naibaf7

I tried to install caffe in odroid-xu4 (Samsung Exynos-5422, ARM Mali-T628 MP6) with clBlas 2.4 (Thanks for @zif520 for providing cmake instruction of clBlas 2.4). I thought I just need one of clBlas and ViennaCL, but not both. However, when cmake, and make, I encountered some fatal errors.

In CMakeLists.txt, I set CPU_ONLY OFF, USE_INDEX_64 OFF, USE_GREENTEA ON, USE_CLBLAS ON, USE_ISAAC OFF, USE_CUDNN OFF

1. However, in cmake/Dependencies.cmake line68, if(USE_GREENTEA) find_package(ViennaCL): it needs ViennaCL. Do I still need ViennaCL, if I just want to use clBlas? After making some changes, it won't execute find_package(ViennaCL).
2. When "make all", it showed "fatal error viennacl/backend/opencl.hpp: No such file or directory" in greentea/greentea.hpp I took a look at greentea folder, it contains "cl_kernels,hpp", "greentea.hpp", "greentea_im2col.hpp" and "greentea_math_functions.hpp". All of them include viennacl header file and use viennacl::ocl::program in greentea_im2col.hpp.

So, if I just want to build with clBlas, do I still need ViennaCL? Or I misunderstand something?

[naibaf7]

@sambookhon ViennaCL provides much more than just the BLAS here. It is also used for device initialization, kernel launching, memory management (partially). Therefore yes, you need ViennaCL in any case. Note however that you never need to link against ViennaCL, it is used header-only (and therefore should not pose any problems).

Let me know if you have more issues. I can also recommend you to switch using https://github.com/BVLC/caffe/tree/opencl, which is my "stable" branch while I experiment more with this branch here.

[sambookhon]

@naibaf7 Thanks for your clarification. I just copy ViennaCL header folder into /usr/local/include so cmake can find it. I almost reached the end of compile, but it showed the error when building CXX object python/CMakeFiles/pycaffe.dir/caffe/_caffe.cpp.o. I used Python 2.7.10 and NumPy 1.8.2. I didn't find useful information on the Internet, could you give me some hint or direction to resolve this problem.

[100%] Building CXX object python/CMakeFiles/pycaffe.dir/caffe/_caffe.cpp.o In file included from /home/odroid/Caffe/caffe-opencl/include/caffe/common.hpp:5:0, from /home/odroid/Caffe/caffe-opencl/include/caffe/blob.hpp:8, from /home/odroid/Caffe/caffe-opencl/include/caffe/caffe.hpp:7, from /home/odroid/Caffe/caffe-opencl/python/caffe/_caffe.cpp:18: /home/odroid/Caffe/caffe-opencl/build/caffe_config.h:43:0: warning: "USE_OPENCV" redefined

define USE_OPENCV

^

:0:0: note: this is the location of the previous definition In file included from /home/odroid/Caffe/caffe-opencl/include/caffe/common.hpp:5:0, from /home/odroid/Caffe/caffe-opencl/include/caffe/blob.hpp:8, from /home/odroid/Caffe/caffe-opencl/include/caffe/caffe.hpp:7, from /home/odroid/Caffe/caffe-opencl/python/caffe/_caffe.cpp:18: /home/odroid/Caffe/caffe-opencl/build/caffe_config.h:44:0: warning: "USE_LEVELDB" redefined #define USE_LEVELDB ^ :0:0: note: this is the location of the previous definition In file included from /home/odroid/Caffe/caffe-opencl/include/caffe/common.hpp:5:0, from /home/odroid/Caffe/caffe-opencl/include/caffe/blob.hpp:8, from /home/odroid/Caffe/caffe-opencl/include/caffe/caffe.hpp:7, from /home/odroid/Caffe/caffe-opencl/python/caffe/_caffe.cpp:18: /home/odroid/Caffe/caffe-opencl/build/caffe_config.h:45:0: warning: "USE_LMDB" redefined #define USE_LMDB ^ :0:0: note: this is the location of the previous definition In file included from /usr/include/python2.7/numpy/arrayobject.h:4:0, from /home/odroid/Caffe/caffe-opencl/python/caffe/_caffe.cpp:11: /home/odroid/Caffe/caffe-opencl/python/caffe/_caffe.cpp: In member function 'PyObject\* caffe::NdarrayCallPolicies::postcall(PyObject_, PyObject_)': /home/odroid/Caffe/caffe-opencl/python/caffe/_caffe.cpp:187:70: error: invalid conversion from 'long int_' to 'npy_intp_ {aka int_}' [-fpermissive] PyObject *arr_obj = PyArray_SimpleNewFromData(num_axes, dims.data(), ^ python/CMakeFiles/pycaffe.dir/build.make:54: recipe for target 'python/CMakeFiles/pycaffe.dir/caffe/_caffe.cpp.o' failed make[2]: *_\* [python/CMakeFiles/pycaffe.dir/caffe/_caffe.cpp.o] Error 1 CMakeFiles/Makefile2:1018: recipe for target 'python/CMakeFiles/pycaffe.dir/all' failed make[1]: **\* [python/CMakeFiles/pycaffe.dir/all] Error 2 Makefile:116: recipe for target 'all' failed make: **\* [all] Error 2

[psyhtest]

@sambookhon

/home/odroid/Caffe/caffe-opencl/python/caffe/_caffe.cpp: In member function 'PyObject* caffe::NdarrayCallPolicies::postcall(PyObject, PyObject)':
/home/odroid/Caffe/caffe-opencl/python/caffe/_caffe.cpp:187:70: error: invalid conversion from 'long int' to 'npy_intp {aka int}' [-fpermissive]
PyObject *arr_obj = PyArray_SimpleNewFromData(num_axes, dims.data(),
^

I think I know this one. I patched it like this:

-    vector<npy_long> dims(blob->shape().begin(), blob->shape().end());
+    vector<npy_int> dims(blob->shape().begin(), blob->shape().end());
     PyObject *arr_obj = PyArray_SimpleNewFromData(num_axes, dims.data(),
                                                   NPY_FLOAT32, data);

@naibaf7 Didn't see it on the caffe master. Is your branch from the caffe dev?

[naibaf7]

@psyhtest This bug is due to the 64 bit indexing feature on this branch. USE_INDEX64 replaces index variables throughout the code (including all kernels) from 4 bytes to 8 bytes. I'll need to investigate a good fix here, because using npy_int will not work with the USE_INDEX64 mode, while npy_long won't work with 32 bit indexing.

It was a necessary fix for huge networks (using indexes over the 2 GB range) that use all of the W9100's 16 GB memory...