search

dividiti / ck-caffe

Collective Knowledge workflow for Caffe to automate installation across diverse platforms and to collaboratively evaluate and optimize Caffe-based workloads across diverse hardware, software and data sets (compilers, libraries, tools, models, inputs):
http://cKnowledge.org
BSD 3-Clause "New" or "Revised" License
193 stars 40 forks source link

"ck compile program:caffe-time-opencl" fails #97

Open psyhtest opened 7 years ago

psyhtest commented 7 years ago 
g++-5 -c    -I../ -DANDROID_USE_OPENMP=ON -DBLAS=Open -DCK_HOST_OS_NAME2_LINUX=1 -DCK_HOST_OS_NAME_LINUX=1 -DCK_TARGET_OS_NAME2_LINUX=1 -DCK_TARGET_OS_NAME_LINUX=1 -DUSE_GREENTEA=ON -DUSE_LMDB=OFF -DUSE_OPENCV=ON -DXOPENME=ON    -I/home/anton/CK_TOOLS/lib-caffe-bvlc-opencl-clblast-trunk-gcc-5.4.0-linux-64/install/include -I/home/anton/CK_TOOLS/lib-gflags-2.2.0-gcc-5.4.0-linux-64/install/include -I/home/anton/CK_TOOLS/lib-glog-development-gcc-5.4.0-linux-64/install/include -I/home/anton/CK_TOOLS/lib-openblas-0.2.19-gcc-5.4.0-linux-64/install/include -I/home/anton/CK_TOOLS/lib-protobuf-host-3.1.0-linux-64/install/include -I/usr/include -I/usr/include -I/home/anton/usr-local/cuda-8.0.61/include -I/usr/include -I/home/anton/CK_TOOLS/lib-viennacl-src-dvdt-master-linux-64/src -I/home/anton/CK_TOOLS/lib-rtl-xopenme-0.3-gcc-5.4.0-linux-64/include -I/home/anton/CK_TOOLS/lib-caffe-bvlc-opencl-clblast-trunk-gcc-5.4.0-linux-64/install/.build_release/src  ../caffe.cpp  -o caffe.o
In file included from /home/anton/CK_TOOLS/lib-caffe-bvlc-opencl-clblast-trunk-gcc-5.4.0-linux-64/install/include/caffe/caffe.hpp:11:0,
                 from ../caffe.cpp:16:
/home/anton/CK_TOOLS/lib-caffe-bvlc-opencl-clblast-trunk-gcc-5.4.0-linux-64/install/include/caffe/filler.hpp: In member function ‘virtual void caffe::GaussianFiller<Dtype>::Fill(caffe::Blob<Dtype>*)’:
/home/anton/CK_TOOLS/lib-caffe-bvlc-opencl-clblast-trunk-gcc-5.4.0-linux-64/install/include/caffe/filler.hpp:95:20: error: ‘is_same’ is not a member of ‘std’
                    std::is_same<int_tp, int32_t>::value ? INT32 : INT64));
                    ^
/home/anton/CK_TOOLS/lib-caffe-bvlc-opencl-clblast-trunk-gcc-5.4.0-linux-64/install/include/caffe/filler.hpp:95:20: note: suggested alternatives:
In file included from /usr/include/boost/type_traits/is_same.hpp:26:0,
                 from /usr/include/boost/range/detail/implementation_help.hpp:16,
                 from /usr/include/boost/range/end.hpp:24,
                 from /usr/include/boost/algorithm/string/trim.hpp:17,
                 from /usr/include/boost/algorithm/string.hpp:19,
                 from ../caffe.cpp:15:
/usr/include/boost/type_traits/is_same.hpp:31:1: note:   ‘boost::is_same’
 BOOST_TT_AUX_BOOL_TRAIT_DEF2(is_same,T,U,false)
 ^
In file included from /home/anton/CK_TOOLS/lib-protobuf-host-3.1.0-linux-64/install/include/google/protobuf/stubs/callback.h:5:0,
                 from /home/anton/CK_TOOLS/lib-protobuf-host-3.1.0-linux-64/install/include/google/protobuf/stubs/common.h:48,
                 from /home/anton/CK_TOOLS/lib-caffe-bvlc-opencl-clblast-trunk-gcc-5.4.0-linux-64/install/include/caffe/proto/caffe.pb.h:9,
                 from /home/anton/CK_TOOLS/lib-caffe-bvlc-opencl-clblast-trunk-gcc-5.4.0-linux-64/install/include/caffe/blob.hpp:10,
                 from /home/anton/CK_TOOLS/lib-caffe-bvlc-opencl-clblast-trunk-gcc-5.4.0-linux-64/install/include/caffe/caffe.hpp:7,
                 from ../caffe.cpp:16:
/home/anton/CK_TOOLS/lib-protobuf-host-3.1.0-linux-64/install/include/google/protobuf/stubs/type_traits.h:109:36: note:   ‘google::protobuf::internal::is_same’
 template <class T, class U> struct is_same;
                                    ^
In file included from /home/anton/CK_TOOLS/lib-caffe-bvlc-opencl-clblast-trunk-gcc-5.4.0-linux-64/install/include/caffe/common.hpp:25:0,
                 from /home/anton/CK_TOOLS/lib-caffe-bvlc-opencl-clblast-trunk-gcc-5.4.0-linux-64/install/include/caffe/blob.hpp:8,
                 from /home/anton/CK_TOOLS/lib-caffe-bvlc-opencl-clblast-trunk-gcc-5.4.0-linux-64/install/include/caffe/caffe.hpp:7,
                 from ../caffe.cpp:16:
/home/anton/CK_TOOLS/lib-caffe-bvlc-opencl-clblast-trunk-gcc-5.4.0-linux-64/install/include/caffe/greentea/greentea.hpp:63:8: note:   ‘caffe::is_same’
 struct is_same {
        ^
In file included from /home/anton/CK_TOOLS/lib-caffe-bvlc-opencl-clblast-trunk-gcc-5.4.0-linux-64/install/include/caffe/caffe.hpp:11:0,
                 from ../caffe.cpp:16:
/home/anton/CK_TOOLS/lib-caffe-bvlc-opencl-clblast-trunk-gcc-5.4.0-linux-64/install/include/caffe/filler.hpp:95:39: error: expected primary-expression before ‘,’ token
                    std::is_same<int_tp, int32_t>::value ? INT32 : INT64));
                                       ^
/home/anton/CK_TOOLS/lib-caffe-bvlc-opencl-clblast-trunk-gcc-5.4.0-linux-64/install/include/caffe/filler.hpp:95:48: error: expected primary-expression before ‘>’ token
                    std::is_same<int_tp, int32_t>::value ? INT32 : INT64));
                                                ^
/home/anton/CK_TOOLS/lib-caffe-bvlc-opencl-clblast-trunk-gcc-5.4.0-linux-64/install/include/caffe/filler.hpp:95:49: error: ‘::value’ has not been declared
                    std::is_same<int_tp, int32_t>::value ? INT32 : INT64));
psyhtest commented 7 years ago

Apparently, std::is_same is a C++11 feature, therefore the fix was simple:

diff --git a/program/caffe-time-opencl/.cm/meta.json b/program/caffe-time-opencl/.cm/meta.json
index 1cd6c37..64d8d64 100644
--- a/program/caffe-time-opencl/.cm/meta.json
+++ b/program/caffe-time-opencl/.cm/meta.json
@@ -109,7 +109,7 @@
     "CK_ENV_LIB_CAFFE_EXTRA_INCLUDE"
   ], 
   "compiler_env": "CK_CXX", 
-  "compiler_flags_as_env": "$<<CK_OPT_UNWIND>>$ $<<CK_ENV_LIB_CAFFE_CXXFLAGS>>$", 
+  "compiler_flags_as_env": "$<<CK_OPT_UNWIND>>$ $<<CK_ENV_LIB_CAFFE_CXXFLAGS>>$ $<<CK_COMPILER_FLAG_CPP11>>$",
   "data_name": "caffe-opencl-time", 
   "extra_ld_vars": "$<<CK_ENV_LIB_HDF5_LFLAG>>$ $<<CK_ENV_LIB_HDF5_LFLAG_HL>>$ $<<CK_ENV_LIB_GLOG_LFLAG>>$ $<<CK_ENV_LIB_BOOST_LFLAG_THREAD>>$ $<<CK_ENV_LIB_BOOST_LFLAG_DATE_TIME>>$ $<<CK_ENV_LIB_BOOST_LFLAG_FILESYSTEM>>$ $<<CK_ENV_LIB_BOOST_LFLAG_SYSTEM>>$ $<<CK_ENV_LIB_OPENCV_LFLAG_IMGPROC>>$ $<<CK_ENV_LIB_OPENCV_LFLAG_HIGHGUI>>$ $<<CK_ENV_LIB_OPENCV_LFLAG_CORE>>$ $<<CK_ENV_LIB_OPENCV_LFLAG_IMGCODECS>>$ $<<CK_EXTRA_LIB_Z>>$ $<<CK_EXTRA_LIB_LOG>>$ $<<CK_EXTRA_LIB_M>>$  $<<CK_ENV_LIB_CAFFE_LFLAG_PROTO>>$ $<<CK_ENV_LIB_CAFFE_LINK_FLAGS>>$ $<<CK_ENV_LIB_STDCPP_STATIC>>$", 
   "main_language": "cpp",
psyhtest commented 7 years ago

Building for Android, however, still fails due to a linking issue :

$ ck compile program:caffe-time-opencl --target_os=android21-arm64
...
aarch64-linux-android-g++ -fPIE -pie -c --sysroot /home/anton/usr-local/android-ndk-r14b/platforms/android-21/arch-arm64    -I../ -DANDROID_USE_OPENMP=ON -DBLAS=Open -DCK_HOST_OS_NAME2_LINUX=1 -DCK_HOST_OS_NAME_LINUX=1 -DCK_TARGET_OS_NAME2_ANDROID=1 -DCK_TARGET_OS_NAME_LINUX=1 -DUSE_GREENTEA=ON -DUSE_LMDB=OFF -DUSE_OPENCV=ON -DXOPENME=ON    -std=c++11 -I/home/anton/CK_TOOLS/lib-caffe-bvlc-opencl-clblast-trunk-android-ndk-4.9.x-android21-arm64/install/include -I/home/anton/CK_TOOLS/lib-gflags-2.2.0-android-ndk-4.9.x-android21-arm64/install/include -I/home/anton/CK_TOOLS/lib-glog-development-android-ndk-4.9.x-android21-arm64/install/include -I/home/anton/CK_TOOLS/lib-openblas-0.2.19-android-ndk-4.9.x-android21-arm64/install/include -I/home/anton/CK_TOOLS/lib-protobuf-3.1.0-android-ndk-4.9.x-android21-arm64/install/include -I/home/anton/CK_TOOLS/lib-opencv-3.2.0-android-ndk-4.9.x-android21-arm64/install/sdk/native/jni/include -I/home/anton/CK_TOOLS/lib-boost-1.62.0-android-ndk-4.9.x-android21-arm64/install/include -I/home/anton/CK_TOOLS/lib-opencl-stubs-1.2-android-ndk-4.9.x-android21-arm64/include -I/home/anton/CK_TOOLS/lib-boost-1.62.0-android-ndk-4.9.x-android21-arm64/install/include -I/home/anton/CK_TOOLS/lib-viennacl-src-dvdt-master-android21-arm64/src -I/home/anton/CK_TOOLS/lib-rtl-xopenme-0.3-android-ndk-4.9.x-android21-arm64/include -I/home/anton/usr-local/android-ndk-r14b/sources/cxx-stl/gnu-libstdc++/4.9/include -I/home/anton/usr-local/android-ndk-r14b/sources/cxx-stl/gnu-libstdc++/4.9/libs/arm64-v8a/include -I/home/anton/CK_TOOLS/lib-caffe-bvlc-opencl-clblast-trunk-android-ndk-4.9.x-android21-arm64/install/.build_release/src  ../caffe.cpp  -o caffe.o
aarch64-linux-android-g++ -fPIE -pie --sysroot /home/anton/usr-local/android-ndk-r14b/platforms/android-21/arch-arm64     caffe.o  -o caffe    -L/home/anton/CK_TOOLS/lib-caffe-bvlc-opencl-clblast-trunk-android-ndk-4.9.x-android21-arm64/install/lib -lcaffe  -L/home/anton/CK_TOOLS/lib-gflags-2.2.0-android-ndk-4.9.x-android21-arm64/install/lib -lgflags -L/home/anton/CK_TOOLS/lib-glog-development-android-ndk-4.9.x-android21-arm64/install/lib  -lglog  -L/home/anton/CK_TOOLS/lib-openblas-0.2.19-android-ndk-4.9.x-android21-arm64/install/lib -lopenblas  -L/home/anton/CK_TOOLS/lib-protobuf-3.1.0-android-ndk-4.9.x-android21-arm64/install/lib -lprotobuf  -L/home/anton/CK_TOOLS/lib-opencv-3.2.0-android-ndk-4.9.x-android21-arm64/install/sdk/native/libs/arm64-v8a -lopencv_core -L/home/anton/CK_TOOLS/lib-boost-1.62.0-android-ndk-4.9.x-android21-arm64/install/lib  -L/home/anton/CK_TOOLS/lib-opencl-stubs-1.2-android-ndk-4.9.x-android21-arm64/lib -lOpenCL  -L/home/anton/CK_TOOLS/lib-rtl-xopenme-0.3-android-ndk-4.9.x-android21-arm64/lib -lrtlxopenme   -lglog -lboost_thread -lboost_date_time -lboost_filesystem -lboost_system -lopencv_imgproc -lopencv_highgui -lopencv_core  -lz -llog -lm    /home/anton/usr-local/android-ndk-r14b/sources/cxx-stl/gnu-libstdc++/4.9/libs/arm64-v8a/libgnustl_static.a
/usr/local/android-ndk-r14b/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/../lib/gcc/aarch64-linux-android/4.9.x/../../../../aarch64-linux-android/bin/ld: warning: libgnustl_shared.so, needed by /home/anton/CK_TOOLS/lib-caffe-bvlc-opencl-clblast-trunk-android-ndk-4.9.x-android21-arm64/install/lib/libcaffe.so, not found (try using -rpath or -rpath-link)
/usr/local/android-ndk-r14b/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/../lib/gcc/aarch64-linux-android/4.9.x/../../../../aarch64-linux-android/bin/ld: caffe: hidden symbol `_ZNSs13_S_copy_charsEPcS_S_' in /home/anton/usr-local/android-ndk-r14b/sources/cxx-stl/gnu-libstdc++/4.9/libs/arm64-v8a/libgnustl_static.a(string-inst.o) is referenced by DSO
/usr/local/android-ndk-r14b/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/../lib/gcc/aarch64-linux-android/4.9.x/../../../../aarch64-linux-android/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
fvella commented 7 years ago

Hi, I fixed the problem putting the flag "-shared" on meta.json. Anyway, I am not using CK variables and the flag is hard-coded in the meta json file.

gfursin commented 7 years ago

I tried to reproduce this issue on a clean machine (velocity, with all clean repos), and it compiles fine (with clblast)! Do you mind to remove CK-TOOLS and ck rm env:* -f and try it from scratch again, please? Can you then tell me if you still have this issue? Because I think that's it's not a problem anymore ... Thanks!

fvella commented 7 years ago

I reproduced the issue on a clean machine. The problem persist if I remove "-shared" from the meta.json of caffe-time-opencl

gfursin commented 7 years ago

Bizzare - which machine did you use? velocity? I again rebuilt everything on velocity without -shared and it worked fine ... However, such problem happens when there is a mix between libs in /usr/lib and compiled by CK via LD_LIBRARY_PATH (I don't yet have a good solution for that) ...

gfursin commented 7 years ago

And do you target x86(64) or it's for Android?

gfursin commented 7 years ago

Oh, I see that it's for Android, while I am checking for x86_64 ...

gfursin commented 7 years ago

I still can't reproduce this issue, but I suggest to remove -shared from meta (it's not correct for Windows), and use the following for compilation on problematic platforms: $ ck compile program:caffe-time-opencl --target_os=android21-arm64 --lflags=-shared