dividiti / ck-caffe

Collective Knowledge workflow for Caffe to automate installation across diverse platforms and to collaboratively evaluate and optimize Caffe-based workloads across diverse hardware, software and data sets (compilers, libraries, tools, models, inputs):
http://cKnowledge.org
BSD 3-Clause "New" or "Revised" License
193 stars 40 forks source link

compilation fails for caffe opencl branch in CPU_ONLY mode #48

Closed DVEfremov closed 7 years ago

DVEfremov commented 7 years ago

Compilation fails for caffe opencl branch in CPU_ONLY mode with error:

/home/daniil/CK-TOOLS/lib-caffe-bvlc-opencl-clblast-trunk-android-ndk-4.9.x-android21-arm-v7a-32/src/src/caffe/layers/softmax_layer.cpp:21:3: error: 'use_slm_' was not declared in this scope
   use_slm_ = (bottom[0]->shape(softmax_axis_) * inner_num_
   ^
make[2]: *** [src/caffe/CMakeFiles/caffe.dir/layers/softmax_layer.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: Leaving directory `/home/daniil/CK-TOOLS/lib-caffe-bvlc-opencl-clblast-trunk-android-ndk-4.9.x-android21-arm-v7a-32/obj'
make[1]: *** [src/caffe/CMakeFiles/caffe.dir/all] Error 2
make[1]: Leaving directory `/home/daniil/CK-TOOLS/lib-caffe-bvlc-opencl-clblast-trunk-android-ndk-4.9.x-android21-arm-v7a-32/obj'
make: *** [all] Error 2
DVEfremov commented 7 years ago

I've fixed error and keep same code for CPU_ONLY mode as for BVLC/caffe master branch

DVEfremov commented 7 years ago

Compilation works fine now but I've got problem at runtime:

...
      I0201 12:38:22.572814 11105 caffe.cpp:411] Use CPU.
...
      I0201 12:05:26.929121  6517 caffe.cpp:426] Performing Backward
      I0201 12:05:26.929172  6517 caffe.cpp:434] *** Benchmark begins ***
      I0201 12:05:26.929227  6517 caffe.cpp:435] Testing for 1 iterations.
      F0201 12:05:26.929330  6517 common.cpp:213] Cannot use GPU in CPU-only Caffe: check mode.
      *** Check failure stack trace: ***
      Aborted 

also /data/local/tmp/tmp/libOpenCL.so still required at runtime for CPU mode

I've tried to create two ck packages with reduced dependencies and turned off flags like GREEN_TEE

and can't see libOpenCL.so at compiled caffelib.so or caffe binaries:

daniil@daniil:~/CK-TOOLS/lib-caffe-bvlc-master-cpu-trunk-android-ndk-4.9.x-android21-arm-v7a-32/install/lib$ readelf -a libcaffe.so | grep NEED
  [ 7] .gnu.version_r    VERNEED         001ddd60 1ddd60 000040 00   A  3   2  4
 0x00000001 (NEEDED)                     Shared library: [libdl.so]
 0x00000001 (NEEDED)                     Shared library: [liblog.so]
 0x00000001 (NEEDED)                     Shared library: [libstdc++.so]
 0x00000001 (NEEDED)                     Shared library: [libm.so]
 0x00000001 (NEEDED)                     Shared library: [libc.so]
 0x6ffffffe (VERNEED)                    0x1ddd60
 0x6fffffff (VERNEEDNUM)                 2
daniil@daniil:~/CK-TOOLS/lib-caffe-bvlc-master-cpu-trunk-android-ndk-4.9.x-android21-arm-v7a-32/install/lib$ ls -l
total 19216
-rw-r--r-- 1 daniil daniil    75324 Feb  1 12:01 libcaffe_jni.so
-rw-r--r-- 1 daniil daniil 18494736 Feb  1 12:00 libcaffe.so
-rw-r--r-- 1 daniil daniil  1098310 Feb  1 11:59 libproto.a
daniil@daniil:~/CK-TOOLS/lib-caffe-bvlc-master-cpu-trunk-android-ndk-4.9.x-android21-arm-v7a-32/install/lib$ 
daniil@daniil:~/CK-TOOLS/lib-caffe-bvlc-master-cpu-trunk-android-ndk-4.9.x-android21-arm-v7a-32/install/bin$ readelf -a caffe | grep NEED
  [ 6] .gnu.version_r    VERNEED         00004a3c 004a3c 000020 00   A  3   1  4
 0x00000001 (NEEDED)                     Shared library: [libcaffe.so]
 0x00000001 (NEEDED)                     Shared library: [libdl.so]
 0x00000001 (NEEDED)                     Shared library: [liblog.so]
 0x00000001 (NEEDED)                     Shared library: [libstdc++.so]
 0x00000001 (NEEDED)                     Shared library: [libm.so]
 0x00000001 (NEEDED)                     Shared library: [libc.so]
 0x6ffffffe (VERNEED)                    0x4a3c
 0x6fffffff (VERNEEDNUM)   
DVEfremov commented 7 years ago

Steps to reproduce at branch issue-48

run new program

not it works without libOpenCL.so:

shell@trelte:/data/local/tmp/tmp $ ls -l *.so                                  
-rw-rw-rw- shell    shell    18494736 2017-02-01 12:46 libcaffe.so

but still got error at run time:

      I0201 12:57:45.596982 13358 caffe.cpp:435] Testing for 1 iterations.
      F0201 12:57:45.597076 13358 common.cpp:213] Cannot use GPU in CPU-only Caffe: check mode.
      *** Check failure stack trace: ***

looks like one more hardcoded case at opencl original branch mode works only in gpu mode

DVEfremov commented 7 years ago

Looks like it because of stub: /caffe/include/caffe/util/device_alternate.hpp used in CPU mode and fails for all *_gpu methods:

#define NO_GPU LOG(FATAL) << "Cannot use GPU in CPU-only Caffe: check mode."
...
void classname<Dtype>::Forward_gpu(const vector<Blob<Dtype>*>& bottom, \
    const vector<Blob<Dtype>*>& top) { NO_GPU; } \
psyhtest commented 7 years ago

@daniil: it's not a bug, it's a feature! :) You can't use any GPU run commands when you compile for the CPU only!

DVEfremov commented 7 years ago

works fine now closing