[Bug] error: model_lib/libmodel_android.a(._lib0.o): not an ELF file

taeyeonlee commented 9 months ago

🐛 Bug

I'm trying build android app but got an error with error: model_lib/libmodel_android.a(._lib0.o): not an ELF file when excuting ./prepare_libs.sh. Whats wrong and how to fix it?

To Reproduce

Steps to reproduce the behavior: https://llm.mlc.ai/docs/deploy/android.html

tvm build successful with libtvm_runtime.so/libtvm.so outputed.
use the defaul file : app-config.json { "model_libs": [ "llama_q4f16_1", "gpt_neox_q4f16_1" ], "model_list": [ { "model_url": "https://huggingface.co/mlc-ai/Llama-2-7b-chat-hf-q4f16_1-MLC/", "local_id": "Llama-2-7b-chat-hf-q4f16_1", "model_lib_path": "libs/Llama-2-7b-chat-hf-q4f16_1-android.tar", "model_lib": "llama_q4f16_1" }, { "model_url": "https://huggingface.co/mlc-ai/RedPajama-INCITE-Chat-3B-v1-q4f16_1-MLC/", "local_id": "RedPajama-INCITE-Chat-3B-v1-q4f16_1", "model_lib_path": "libs/RedPajama-INCITE-Chat-3B-v1-q4f16_1-android.tar", "model_lib": "gpt_neox_q4f16_1" } ] }
download the prebuilt lib : from https://github.com/mlc-ai/binary-mlc-llm-libs/tree/main ./dist/libs/Llama-2-7b-chat-hf-q4f16_1-android.tar ./dist/libs/RedPajama-INCITE-Chat-3B-v1-q4f16_1-android.tar
excute ./prepare_libs.sh
the error is following.

taeyeonlee@taeyeonlee-15U50Q-SP7PL:~/mlc-llm/android/library$ ./prepare_libs.sh

rustup target add aarch64-linux-android info: component 'rust-std' for target 'aarch64-linux-android' is up to date
mkdir -p build/model_lib
python3 prepare_model_lib.py Creating lib from ['/home/taeyeonlee/mlc-llm/dist/libs/Llama-2-7b-chat-hf-q4f16_1-android.tar', '/home/taeyeonlee/mlc-llm/dist/libs/RedPajama-INCITE-Chat-3B-v1-q4f16_1-android.tar']..
cd build
touch config.cmake
'[' /home/taeyeonlee/mlc-llm/3rdparty/tvm -ne 0 ']' ./prepare_libs.sh: line 12: [: /home/taeyeonlee/mlc-llm/3rdparty/tvm: integer expression expected
cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE=/home/taeyeonlee/Android/Sdk/ndk/26.1.10909125/build/cmake/android.toolchain.cmake -DCMAKE_INSTALL_PREFIX=. -DCMAKE_CXX_FLAGS=-O3 -DANDROID_ABI=arm64-v8a -DANDROID_NATIVE_API_LEVEL=android-24 -DANDROID_PLATFORM=android-24 -DCMAKE_FIND_ROOT_PATH_MODE_PACKAGE=ON -DANDROID_STL=c++_static -DUSE_HEXAGON_SDK=OFF -DMLC_LLM_INSTALL_STATIC_LIB=ON -DCMAKE_SKIP_INSTALL_ALL_DEPENDENCY=ON -DUSE_OPENCL=ON -DUSE_CUSTOM_LOGGING=ON -- Hide private symbols -- TVM_HOME: 3rdparty/tvm -- Hide private symbols... -- Forbidding undefined symbols in shared library, using -Wl,--no-undefined on platform Android -- Building for Android -- Didn't find the path to CCACHE, disabling ccache -- VTA build with VTA_HW_PATH=/home/taeyeonlee/mlc-llm/3rdparty/tvm/3rdparty/vta-hw -- Build VTA runtime with target: sim -- Enabled runtime search for OpenCL library location -- Couldn't build OpenCL-Gtests -- Android Vulkan_INCLUDE_DIRS=/home/taeyeonlee/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include -- Skip finding SPIRV in Android, make sure you only build tvm runtime. -- Build with contrib.random -- Build with contrib.sort -- Build with contrib.hybriddump -- Git found: /usr/bin/git -- Found TVM_GIT_COMMIT_HASH=7c4d629ae6600da6c8930758e3175a93a88be458 -- Found TVM_GIT_COMMIT_TIME=2024-01-08 14:22:16 -0500 -- Building with TVM Map... -- Build with thread support... -- CLANG_VERSION 17.0 -- Setting enhanced clang warning flags -- Added "-fuse-ld=lld" to linker flags -static-libstdc++ -Wl,--build-id=sha1 -Wl,--no-rosegment -Wl,--no-undefined-version -Wl,--fatal-warnings -Wl,--no-undefined -Qunused-arguments -- Build without FlashInfer -- system-nameAndroid -- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) -- VERSION: 0.2.00 CMake Warning at /home/taeyeonlee/mlc-llm/CMakeLists.txt:105 (message): Cannot find libflash_attn. The model must not have been built with --use-flash-attn-mqa option.

-- TVM_HOME: /home/taeyeonlee/mlc-llm/android/library/../../3rdparty/tvm -- JNI_INCLUDE_DIRS=/usr/lib/jvm/java-17-openjdk-amd64/include;/usr/lib/jvm/java-17-openjdk-amd64/include/linux;/usr/lib/jvm/java-17-openjdk-amd64/include -- JNI_LIBRARIES=/usr/lib/jvm/java-17-openjdk-amd64/lib/libjawt.so;/usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so -- Configuring done -- Generating done -- Build files have been written to: /home/taeyeonlee/mlc-llm/android/library/build

make tvm4j_runtime_packed -j10 Consolidate compiler generated dependencies of target tvm_libinfo_objs Consolidate compiler generated dependencies of target tokenizer_cpp_objs Consolidate compiler generated dependencies of target mlc_llm_objs [ 8%] Built target tvm4j_core [ 8%] Built target tokenizers_c [ 8%] Built target tvm_libinfo_objs [ 8%] Built target tokenizer_cpp_objs [ 16%] Built target mlc_llm_objs Consolidate compiler generated dependencies of target sentencepiece-static Consolidate compiler generated dependencies of target tvm_runtime_objs [ 50%] Built target sentencepiece-static [ 50%] Built target tokenizers_cpp [100%] Built target tvm_runtime_objs [100%] Built target tvm_runtime [100%] Built target mlc_llm_static Consolidate compiler generated dependencies of target tvm4j_runtime_packed [100%] Building CXX object CMakeFiles/tvm4j_runtime_packed.dir/home/taeyeonlee/mlc-llm/3rdparty/tvm/jvm/native/src/main/native/org_apache_tvm_native_c_api.cc.o [100%] Linking CXX shared library libtvm4j_runtime_packed.so ld.lld: error: model_lib/libmodel_android.a(._lib0.o): not an ELF file clang++: error: linker command failed with exit code 1 (use -v to see invocation) make[3]: [CMakeFiles/tvm4j_runtime_packed.dir/build.make:104: libtvm4j_runtime_packed.so] Error 1 make[2]: [CMakeFiles/Makefile2:268: CMakeFiles/tvm4j_runtime_packed.dir/all] Error 2 make[1]: [CMakeFiles/Makefile2:275: CMakeFiles/tvm4j_runtime_packed.dir/rule] Error 2 make: [Makefile:203: tvm4j_runtime_packed] Error 2 taeyeonlee@taeyeonlee-15U50Q-SP7PL:~/mlc-llm/android/library$

Expected behavior

build successful

Environment

Platform : Android
Operating system : Ubuntu
Device : (e.g. iPhone 12 Pro, PC+RTX 3090, ...)
How you installed MLC-LLM : source
How you installed TVM-Unity : pip
Python version : 3.10.12
GPU driver version : NVIDIA-SMI 545.23.08 Driver Version: 545.23.08
CUDA/cuDNN version : 12.3
TVM Unity Hash Tag (python -c "import tvm; print('\n'.join(f'{k}: {v}' for k, v in tvm.support.libinfo().items()))", applicable if you compile models): USE_NVTX: OFF USE_GTEST: AUTO SUMMARIZE: OFF USE_IOS_RPC: OFF USE_MSC: OFF USE_ETHOSU: CUDA_VERSION: NOT-FOUND USE_LIBBACKTRACE: AUTO DLPACK_PATH: 3rdparty/dlpack/include USE_TENSORRT_CODEGEN: OFF USE_THRUST: OFF USE_TARGET_ONNX: OFF USE_AOT_EXECUTOR: ON BUILD_DUMMY_LIBTVM: OFF USE_CUDNN: OFF USE_TENSORRT_RUNTIME: OFF USE_ARM_COMPUTE_LIB_GRAPH_EXECUTOR: OFF USE_CCACHE: AUTO USE_ARM_COMPUTE_LIB: OFF USE_CPP_RTVM: USE_OPENCL_GTEST: /path/to/opencl/gtest USE_MKL: OFF USE_PT_TVMDSOOP: OFF MLIR_VERSION: NOT-FOUND USE_CLML: OFF USE_STACKVM_RUNTIME: OFF USE_GRAPH_EXECUTOR_CUDA_GRAPH: OFF ROCM_PATH: /opt/rocm USE_DNNL: OFF USE_VITIS_AI: OFF USE_MLIR: OFF USE_RCCL: OFF USE_LLVM: llvm-config --ignore-libllvm --link-static USE_VERILATOR: OFF USE_TF_TVMDSOOP: OFF USE_THREADS: ON USE_MSVC_MT: OFF BACKTRACE_ON_SEGFAULT: OFF USE_GRAPH_EXECUTOR: ON USE_NCCL: OFF USE_ROCBLAS: OFF GIT_COMMIT_HASH: 7c4d629ae6600da6c8930758e3175a93a88be458 USE_VULKAN: ON USE_RUST_EXT: OFF USE_CUTLASS: OFF USE_CPP_RPC: OFF USE_HEXAGON: OFF USE_CUSTOM_LOGGING: OFF USE_UMA: OFF USE_FALLBACK_STL_MAP: OFF USE_SORT: ON USE_RTTI: ON GIT_COMMIT_TIME: 2024-01-08 14:22:16 -0500 USE_HEXAGON_SDK: /path/to/sdk USE_BLAS: none USE_ETHOSN: OFF USE_LIBTORCH: OFF USE_RANDOM: ON USE_CUDA: OFF USE_COREML: OFF USE_AMX: OFF BUILD_STATIC_RUNTIME: OFF USE_CMSISNN: OFF USE_KHRONOS_SPIRV: OFF USE_CLML_GRAPH_EXECUTOR: OFF USE_TFLITE: OFF USE_HEXAGON_GTEST: /path/to/hexagon/gtest PICOJSON_PATH: 3rdparty/picojson USE_OPENCL_ENABLE_HOST_PTR: OFF INSTALL_DEV: OFF USE_PROFILER: ON USE_NNPACK: OFF LLVM_VERSION: 15.0.7 USE_OPENCL: OFF COMPILER_RT_PATH: 3rdparty/compiler-rt RANG_PATH: 3rdparty/rang/include USE_SPIRV_KHR_INTEGER_DOT_PRODUCT: OFF USE_OPENMP: OFF USE_BNNS: OFF USE_CUBLAS: OFF USE_METAL: OFF USE_MICRO_STANDALONE_RUNTIME: OFF USE_HEXAGON_EXTERNAL_LIBS: OFF USE_ALTERNATIVE_LINKER: AUTO USE_BYODT_POSIT: OFF USE_HEXAGON_RPC: OFF USE_MICRO: OFF DMLC_PATH: 3rdparty/dmlc-core/include INDEX_DEFAULT_I64: ON USE_RELAY_DEBUG: OFF USE_RPC: ON USE_TENSORFLOW_PATH: none TVM_CLML_VERSION: USE_MIOPEN: OFF USE_ROCM: OFF USE_PAPI: OFF USE_CURAND: OFF TVM_CXX_COMPILER_PATH: /opt/rh/gcc-toolset-11/root/usr/bin/c++ HIDE_PRIVATE_SYMBOLS: ON
Additional context

BakingBrains commented 9 months ago

Any solution here?

BakingBrains commented 9 months ago

I facing similar issue

[ 93%] Building CXX object mlc_llm/CMakeFiles/mlc_llm_objs.dir/cpp/streamer.cc.o
[ 93%] Building CXX object mlc_llm/CMakeFiles/mlc_llm_objs.dir/cpp/tokenizers.cc.o
1 warning generated.
[ 93%] Built target mlc_llm_objs
[ 93%] Built target tvm_runtime_objs
make[1]: *** [CMakeFiles/Makefile2:275: CMakeFiles/tvm4j_runtime_packed.dir/rule] Error 2
make: *** [Makefile:203: tvm4j_runtime_packed] Error 2

Any suggestions here?

Thank you

BakingBrains commented 9 months ago

@taeyeonlee any solution here?

Facing the same error

ld.lld: error: model_lib/libmodel_android.a(._lib0.o): not an ELF file
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
make[3]: *** [CMakeFiles/tvm4j_runtime_packed.dir/build.make:104: libtvm4j_runtime_packed.so] Error 1
make[2]: *** [CMakeFiles/Makefile2:268: CMakeFiles/tvm4j_runtime_packed.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:275: CMakeFiles/tvm4j_runtime_packed.dir/rule] Error 2
make: *** [Makefile:203: tvm4j_runtime_packed] Error 2

Kartik14 commented 9 months ago

@taeyeonlee @BakingBrains This should fix it, can you please try to download the prebuilt libs again and check?

BakingBrains commented 9 months ago

@Kartik14 It's working thanks

mlc-ai / mlc-llm