search

dividiti / ck-caffe

Collective Knowledge workflow for Caffe to automate installation across diverse platforms and to collaboratively evaluate and optimize Caffe-based workloads across diverse hardware, software and data sets (compilers, libraries, tools, models, inputs):
http://cKnowledge.org
BSD 3-Clause "New" or "Revised" License
193 stars 40 forks source link

"ck compile program:caffe-time --target_os=android21-arm" fails with Android NDK GCC compiler #51

Open psyhtest opened 7 years ago

psyhtest commented 7 years ago 
$ ck compile program:caffe-time --target_os=android21-arm

<...>

In file included from /home/anton/CK_TOOLS/lib-opencv-development-android-ndk-4.9.x-android21-arm-32/src/3rdparty/libwebp/cpu-features/cpu-features.c:61:0:android-ndk-r13b/platforms/android-21/arch-arm/usr/include/machine/cpu-features.h:52:6: error: #error Unknown or unsupported ARM architecture
 #    error Unknown or unsupported ARM architecture
      ^
make[2]: *** [3rdparty/libwebp/CMakeFiles/libwebp.dir/cpu-features/cpu-features.c.o] Error 1
make[2]: *** Waiting for unfinished jobs....

Full log: caffe-time.android21-arm.gcc.txt

psyhtest commented 7 years ago

A trivial workaround seems to use:

ck compile program:caffe-time --target_os=android21-arm-v7a

but one would naturally expect:

ck compile program:caffe-time --target_os=android21-arm

to succeed if it's allowed.

gfursin commented 7 years ago

Well, this is a real program (and not program:caffe which is a front-end to already compiled caffe), so you need to compile it first ;). Therefore it's not a workaround, but correct usage ;) ... I also wrote it in the docs ...

As for OS - originally I had only -arm, but then it caused lots of confusion, because different packages means different things under default -arm (I noticed it when I was unified all Caffe deps), that's why eventually I tried to unify it similar to what other meant by this:

Also, notice, that if we use device module for ARM's workload automation, it will try to detect device features and will try to automatically pre-set correct OS ...

So, it's more about conventions. Otherwise, how will we specify default ABI of "armeabi"? Which CK OS extension should we use?

psyhtest commented 7 years ago

I see what you mean. But I used the compile action in both cases: just with --target_os=android21-arm-v7a it succeeded and with --target_os=android21-arm failed halfway...

I guess I am complaining that CK should have given a warning and presented a list of options for the ambiguous target arm. In fact, it would be nice to use something like:

$ ck compile program:caffe-time --target_tags=android,arm

and then be asked to resolve the target with supported versions of Android and architecture...

gfursin commented 7 years ago

My idea was to keep basic blocks such as "program" quite simple and gradually add more functionality in a higher-level modules. As I mentioned, there is already additional functionality which we do not use much yet, where you connect your "machine", describe it, detect parameters, and then use it as a target, i.e. $ ck add machine:my-target-machine above command should normally be able to detect the most appropriate Android OS version and ABI, and then you can use it as following: $ ck compile program:caffe-time --speed --target=my-target-machine

This is the preffered method, but I just didn't have time to thoroughly test it though. It was prepared as a part of CK-WA: https://github.com/ctuning/ck-wa

However, eventually, we can also enhance compilation with tags, as you suggest. I am neutral to that, particularly if it is useful for users and simple to implement ;) ...