Open xumengwei opened 6 years ago
@arank could you help diagnose the issue?
I haven't looked at Amalgamation for android in a while, or run across this issue but from what I can tell it looks like you are missing the library includes in the mxnet_predict-all.cc file that has the utility functions to open file descriptors and link them to objects in memory. https://linux.die.net/man/3/shm_open maybe ensuring the right imports from the man pages here will help.
Hi @arank, I checked that my generated mxnet_predict-all.cc includes the libraries required for shm_open: sys/mman.h, sys/stat.h, and fcntl.h. So it's really confusing.
This issue come form cpu_shared_storage (17 nov. 2017), shm_open
and shm_unlink
are deprecated on Android and are replaced by ASHMEM library
Is there a way to disable it ?
@piiswrong
after this commit c8f7dce0eb49ab1a62ddc2c7e37b93e9b92c2ae4 there is also issues with packet type
can you use docker_multiarch/Dockerfile.build.android.armv7 instead?
@larroy with your dockerfile it's regular mxnet lib ? I think amalgamation has different API and is lighter.
We are planning to remove amalgamation, can you elaborate on why it's lighter?
As there is only the prediction API, I think the resulting .so was lighter than the full mxnet library ?
@edmBernard can you provide hard numbers? this would be useulf. I don't think that's the case as it compiles everything, except a few blacklisted headers.
on my computer I got this:
17M libmxnet_predict.so <-- amalgamation result
47M libmxnet.so <-- full mxnet
I will try to use tvm, It seem better to use right know as there is even hardware optimisation.
That's interesting. Could you share a list of symbols using nm? for example so we can diff?
nm of my mxnet_predict.so : https://gist.github.com/edmBernard/2c858a3b9c04e9f3072763936e1011e3
Can you paste the full one for reference? Thank you!
nm of my mxnet.so https://gist.github.com/edmBernard/c74b5f939f261cb819b36bdc7617eb29
Thanks, basically I think you are compiling more features like KVStore on the big one, plus one seems to be 64bit and the other 32bit, no wonder is bigger. There's some ndarray symbols missing on the amalgamation build which I'm not sure why.
can you disable kvstore, cuda, cblas, mkl, opencv, etc? mkdir -p build && cd build cmake -DUSE_CPP_PACKAGE=ON -DUSE_CUDA=OFF -DUSE_OPENMP=OFF -DUSE_OPENCV=OFF -DCMAKE_BUILD_TYPE=Debug -GNinja .. ninja -v
also disable other stuff in CMakeLists.txt....
your are right when building full mxnet I forgot to remove gcc in compilator configuration. But now I got this error :(
/home/dev/lib/arm7-toolchain/bin/../lib64/clang/3.8.275480/include/mmintrin.h:366:40: error: cannot initialize a parameter of type 'int' with an rvalue of
type '__v8qi' (vector of 8 'char' values)
return (__m64)__builtin_ia32_paddb((__v8qi)__m1, (__v8qi)__m2);
If you're interested I have dockerfile to build amalgamation for android here
git clone https://github.com/edmBernard/DockerFiles.git
cd DockerFiles
make mxnet_android
I will try to build amalgamation with your Dockerfile as in mine I use clang instead of gcc
We also use clang for android, won't compile without clang due to stl.
I compiled a libmxnet_predict.so, but in android studio there is " A/libc: Fatal signal 4 (SIGILL), code 1, fault addr 0xca1009b8 in tid 29855 " fault.
How did you use it in Android studio ?
I use libmxnet_predict.so
to build C++ library and integrate it in Java project.
If you want to directly integrate amalgamation in java you can use the other .so file (don't remember the exact filename but something like jni_mxnet_prediction.so
)
@someoneAlready I meet the same problem,have you solve this?
@billhyde I use docker to build the libmxnet.so, which works well.
@someoneAlready Thank you for your reply, I will try it. Thanks
@someoneAlready The docker's link is dead?
There is some docker with cross compilation for android in CI https://github.com/apache/incubator-mxnet/tree/master/ci/docker (I don't have test them)
The initial problem should have been fixed by https://github.com/apache/incubator-mxnet/pull/9898. You can try compiling for Android with running ci/build.py -p android_armv7
. If there is no issue anymore you could resolve the ticket.
Environment: MacOS, target: Nexus 6
I built the toolchain as follows
python build/tools/make_standalone_toolchain.py --arch arm --api 21 --install-dir /tmp/my-android-toolchain --stl=libc++ --force
And openblas builds successfully. Then I tried to build amalgamation as follows
Then comes the error
/Users/echo/GoogleDrive/DL/mxnet/amalgamation/../src/storage/./cpu_shared_storage_manager.h:30:10: fatal error: 'sys/fcntl.h' file not found
I replace the #include <sys/fcntl.h> with #include, and also add the following DEP to MAKEFILE
Now the expanding works fine, but the compilation still breaks as follows
Is there anybody can help me out of this? I've been stuck on this for a long period (: