[Bug] Unable to build and run the android app from source

[kmadsen]

🐛 Bug

Previously, I was able to build the app from source and then it would crash upon initializing the chat with the downloaded weights.

Now with the latest changes, I'm no longer able to build the app from source.

To Reproduce

I've attached a Docker project with multiple stages. This is a simplified version of what I'm using to build the app.

mlc-android-docker-builder.zip

Steps to reproduce the behavior:

1. Make sure to have Docker on your machine
2. Download the zip and extract it to your machine
3. cd mlc-android-docker-builder
4. sh run.sh

Expected behavior

The mlc-android project should produce an android app that can be used to build an apk from source.

Environment

• Build has been tested on: M1 Mac, Windows 11 x86, and Ubuntu 22.04
• Target Android device: Samsung Tab S8+
• How you installed project: All details in the Dockerfiles

Additional context

I've also tried the option to download TVM Unity through https://mlc.ai/package/. But continue to have issues there.

Here is the output of the build from the mlc-android-docker-builder. It depends on the current HEAD of this repository, so if everything is fixed it seems the docker files may work too (potentially need some edits)

When running the `prepare_libs.sh` script, it fails with the following error:

--snip--
[ 90%] Building CXX object tvm/CMakeFiles/tvm_runtime_objs.dir/src/runtime/opencl/opencl_wrapper/opencl_wrapper.cc.o
[ 90%] Building CXX object tvm/CMakeFiles/tvm_runtime_objs.dir/src/runtime/contrib/random/random.cc.o
[ 90%] Building CXX object tokenizers/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/util.cc.o
[100%] Building CXX object tvm/CMakeFiles/tvm_runtime_objs.dir/src/runtime/contrib/sort/sort.cc.o
[100%] Building CXX object tokenizers/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/word_model.cc.o
[100%] Building CXX object tokenizers/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/absl/flags/flag.cc.o
[100%] Linking CXX static library libsentencepiece.a
[100%] Built target sentencepiece-static
[100%] Linking CXX static library libtokenizers_cpp.a
[100%] Built target tokenizers_cpp
[100%] Built target tvm_runtime_objs
[100%] Linking CXX static library libtvm_runtime.a
You have build static version of the TVM runtime library. Make sure to use --whole-archive when linking it into your project.
[100%] Built target tvm_runtime
[100%] Linking CXX static library libmlc_llm.a
[100%] Built target mlc_llm_static
[100%] Linking CXX executable mlc_chat_cli
[100%] Built target mlc_chat_cli
+ cmake --build . --target install --config release -j
Install the project...
-- Install configuration: "Release"
-- Installing: /app/mlc-llm/android/build/lib/libmlc_llm.a
-- Installing: /app/mlc-llm/android/build/lib/libtokenizers_cpp.a
-- Installing: /app/mlc-llm/android/build/lib/libsentencepiece.a
-- Installing: /app/mlc-llm/android/build/lib/libtvm_runtime.a
-- Installing: /app/mlc-llm/android/build/lib/libtokenizers_c.a
+ cd ..
+ rm -rf build/tvm_home
+ ln -s /app/relax build/tvm_home
+ python prepare_model_lib.py
./prepare_libs.sh: line 31: python: command not found
root@9e791c994a98:/app/mlc-llm/android# exit
exit

[twiceyuan]

Build failed with a exception:

> Task :app:buildJni FAILED
make: Entering directory 'path-to-mlc-llm/android/MLCChat/app/src/main/jni'
fcntl(): Bad file descriptor
Android NDK: WARNING: APP_PLATFORM android-24 is higher than android:minSdkVersion 1 in path-to-mlc-llm/android/MLCChat/app/src/main/AndroidManifest.xml. NDK binaries will *not* be compatible with devices older than android-24. See https://android.googlesource.com/platform/ndk/+/master/docs/user/common_problems.md for more information.
make: Leaving directory 'path-to-mlc-llm/android/MLCChat/app/src/main/jni'
make: Entering directory 'path-to-mlc-llm/android/MLCChat/app/src/main/jni'
[arm64-v8a] Compile++      : tvm4j_runtime_packed <= org_apache_tvm_native_c_api.cc
path-to-mlc-llm/android/MLCChat/app/src/main/jni/org_apache_tvm_native_c_api.cc:25:10: fatal error: 'tvm_runtime.h' file not found
#include "tvm_runtime.h"
         ^~~~~~~~~~~~~~~
1 error generated.
make: *** [path-to-ndk-home/build/core/build-binary.mk:424: path-to-mlc-llm/android/MLCChat/app/src/main/obj/local/arm64-v8a/objs/tvm4j_runtime_packed/org_apache_tvm_native_c_api.o] Error 1
make: Leaving directory 'path-to-mlc-llm/android/MLCChat/app/src/main/jni'

[twiceyuan]

I also tried restoring tvm_runtime.h which was removed in this commit and rerunning the buildJni task, now another error:

ld: error: path-to-mlc-llm/android/MLCChat/app/src/main/jni/../../../../../build/lib/libmodel_android.a(RedPajama-INCITE-Chat-3B-v1-q4f16_0-android.tar): not an ELF file
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [path-to-ndk-home/build/core/build-binary.mk:657: path-to-mlc-llm/android/MLCChat/app/src/main/obj/local/arm64-v8a/libtvm4j_runtime_packed.so] Error 1

I confirmed the type of the file corresponding to the path libmodel_android.a in the error message, and the output is as follows:

$ file path-to-mlc-llm/android/MLCChat/app/src/main/jni/../../../../../build/lib/libmodel_android.a
path-to-mlc-llm/android/MLCChat/app/src/main/jni/../../../../../build/lib/libmodel_android.a: current ar archive

[Mgz-97]

also meet this issue, have your commit fixed this issue? @twiceyuan

[kmadsen]

For what it's worth. We ended up reverting back to an older version of mlc-llm to make our progress.

Also this is the tvm_runtime.h. After you build the android app you can create it manually and put it inside as app/src/main/jni/tvm_runtime.h

#define DMLC_USE_LOGGING_LIBRARY <tvm/runtime/logging.h>
#define TVM_USE_LIBBACKTRACE 0
#include <dlfcn.h>
#include <dmlc/logging.h>
#include <dmlc/thread_local.h>
#include <tvm/runtime/c_runtime_api.h>

[kmadsen]

Closing this one as many things have changed and documentation is a bit better