search

dividiti / ck-caffe

Collective Knowledge workflow for Caffe to automate installation across diverse platforms and to collaboratively evaluate and optimize Caffe-based workloads across diverse hardware, software and data sets (compilers, libraries, tools, models, inputs):
http://cKnowledge.org
BSD 3-Clause "New" or "Revised" License
194 stars 41 forks source link

CMake error: Android: No CMAKE_SYSROOT was selected. #141

Closed udexon closed 5 years ago

udexon commented 5 years ago

https://github.com/ctuning/ck/issues/97

This error occurs at 2 computers I have tested. Both are x86_64 Ubuntu 18.04. Any idea how to solve it? Thank you very much.

$ uname -a
Linux hongwu-Dell-System-XPS-15Z 4.15.0-29-generic #31-Ubuntu SMP Tue Jul 17 15:39:52 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

$ ck compile program:caffe-classification --target_os=android21-arm64
....
patching file examples/CMakeLists.txt
patching file tools/CMakeLists.txt
/home/hongwu/CK/ck-caffe/package/lib-caffe-bvlc-opencl-cpu-universal/patch.android/android.fgg.patch2
patching file src/caffe/CMakeLists.txt
Hunk #1 succeeded at 132 with fuzz 2 (offset 107 lines).

Cleaning ...

Executing extra script ...
**************************************************************
Preparing vars for Caffe ...
CMake Error at /usr/share/cmake-3.10/Modules/Platform/Android-Initialize.cmake:39 (message):
  Android: No CMAKE_SYSROOT was selected.
Call Stack (most recent call first):
  /usr/share/cmake-3.10/Modules/CMakeSystemSpecificInitialize.cmake:8 (include)
  CMakeLists.txt:18 (project)

-- Configuring incomplete, errors occurred!
Error: cmake failed!
gfursin commented 5 years ago

Hi @udexon - thank you for reporting. I just checked the problem and see several issues.

First of all you selected a strange package "lib-caffe-bvlc-opencl-cpu-universal" for compilation - I don't remember what is that (@psyhtest ?) and it looks like it doesn't support Android. So I marked it as only for Linux/MacOS or Windows.

Second, I am not sure if Caffe master continues supporting Android and the latest NDK - I tried to compile it for Android with latest NDK r18b and it fails. That's why you can still use our stable package "lib-caffe-bvlc-master-cpu-universal-20171013" with NDK r13b - that works. You need to do the following to test it.

First of all, update CK repos (to fetch my latest update) and clean your environment so that you can use new NDK:

$ ck pull all --kernel
$ ck clean env:* -f

Then get NDK r13b and install it somewhere. Then detect it by CK as follows:

$ ck detect soft:compiler.gcc.android.ndk --target_os=android21-arm64

Now you can install stable Caffe for CPU as follows:

$ ck install package:lib-caffe-bvlc-master-cpu-universal-20171013 --reuse_deps --target_os=android21-arm64

It should force various stable sub-dependencies and you can select Boost 1.64 (I tested it and it works with that version)

Then you should be able to compile and run your classification example:

$ ck compile program:caffe-classification --target_os=android21-arm64
$ ck run program:caffe-classification --target_os=android21-arm64

As for using newer NDK, unfortunately, Caffe conflicts with new libraries - it's the problem of Caffe, not CK, and sometimes we fix that by providing CK patches to the package or asking Caffe colleagues to fix problems ... And I don't know how it will work with the latest one without GCC there - I suspect lots of incompatibilities ...

Please tell me if it works - we can then close this ticket! Thanks a lot for your feedback.

udexon commented 5 years ago

Thank you for the prompt reply!!

I was merely trying out the example on https://github.com/dividiti/ck-caffe/wiki/Installation under Android section: $ ck compile program:caffe-classification --target_os=android21-arm64

I will try your suggestions.

What I intended to do is to use OpenCL on Android -- I suppose you are aware that Google does not officially support this.

I have read about it on ARM Compute Library (github and ARM's own website).

The only other post I read about successful case of OpenCL Android is https://github.com/gcp/leela-zero/issues/1679.

Also, there is a bazel sample which I tested partially: https://github.com/supernovaremnant/bazel-android-opencl

The trouble with bazel is that the spec changed too fast -- I had to use bazel-0.14 for the old repo to work.

I suppose ck and bazel are quite similar. So I mention this for your reference. I hope you find it useful.

Please let me know if there is any area that you need help on. Our team is really interested in working with others to find solutions for Android OpenCL.

We also work with Forth community comp.lang.forth (Newsgroup) on this matter.

I am sure you realize the potential benefits of being able to run OpenCL on Android with Mali GPU? This has been a long running issue for several years.

We were hoping that Android OpenCL would be a boost for cryptocurrency mining -- which I am sure you are aware that it is a huge business.

psyhtest commented 5 years ago

We were hoping that Android OpenCL would be a boost for cryptocurrency mining

Do you have specific OpenCL programs in mind that you wish to port to Android? How would you distribute them, via Google Play?

udexon commented 5 years ago

We were hoping that Android OpenCL would be a boost for cryptocurrency mining

Do you have specific OpenCL programs in mind that you wish to port to Android? How would you distribute them, via Google Play?

There are several things need to be done before we can roll out our project as an Android app.

Firstly, we aim to enable OpenCL programming in an Android app like GForth (github, Play Sotre)

(If you are not familiar Forth, please visit comp.lang.forth, FORTH PROGRAMMING LANGUAGE 21st CENTURY.)

Secondly, we need to port existing cryptocurrency code to Forth. This may take a few more months at least.

Of course, there is interest from investors as we speak -- which hopefully will speed things up if more funds are available.

I think Forth community has many age tested (50 years old!!) wisdoms that modern project like yours can benefit from -- especially in the area of graph theory. There are plenty of 60 years old ++ programmers and engineers in the forums I posted.

udexon commented 5 years ago

ck run program:caffe-classification --target_os=android21-arm64

@gfursin

Thanks a lot! It is working now.

Just need to make sure I choose the correct target platform now.

Thanks again for the amazing. Do let me know if I can contribute in any way.

gfursin commented 5 years ago

You are welcome, @udexon !

Though CK and Basel can be used to build packages, CK is more a workflow framework with high-level Python APIs to automate, reproduce and share experiments - package building is just one (though essential) part of it ;) ...

By the way, if you have issues or feedback about CK, feel free to use our CK discussion group: https://groups.google.com/forum/#!forum/collective-knowledge

I now don't work so much with OpenCL - I see that @psyhtest is following up on that - he is our OpenCL guru ;) ! As for other contributions: we more-or-less finished our proof-of-concept stage for CK this year and now raising funding to improve documentation (APIs, "how to" guides, contributor guides), provide more automatic tests, and share more components ... We plan to discuss it at the above CK mailing list!

Once again thanks a lot for your positive feedback!

psyhtest commented 5 years ago

@udexon I used to lead the Mali OpenCL compiler team, so I would be pleased if OpenCL was used more on Android.

If you are not familiar Forth

Oh, I know Forth. My classmate taught himself Forth some 25 years ago! (His OS choice was similarly unconventional: he played with OS/2 Warp when Windows 95 was all the rage.:))

we need to port existing cryptocurrency code to Forth. This may take a few more months at least.

How do you plan to execute Forth code on Android: are there cross-compilers available? Also, OpenCL is a C/C++ API. Using it from Forth might not be trivial.

I mean, everything is possible with funds :) but you may be looking at several person-years of work in addition to your crypto code porting effort. Is it really worth it?

udexon commented 5 years ago

@psyhtest Here's a short article of mine, I hope you find it interesting: https://www.linkedin.com/pulse/multitiered-stack-machine-nsm-using-5gl-fifth-graph-ng-ph-d-/

It will be a long answer to address your comments -- which I am glad to. But I do not want to be rude to take up space here. I suggest you contact me via LinkedIn (as shown above). There is also some confidential information which I do not mind sharing with technical peers like yourself, but perhaps not here.

psyhtest commented 5 years ago

@udexon Thanks. I now have more questions (e.g. what's the connection between stack machines, graphs, cryptomining) but sadly no time to pursue this further at this time...

udexon commented 5 years ago

@psyhtest My latest post on comp.lang.forth may answer some of your questions.

You are welcome to alert / contact me via LinkedIn or post a question in my github repos -- should you want to find out latest updates.