[Bug] error: TVM runtime cannot find vm_load_executable when run MLCChat on MyMac(Designed for iPad) for codeLlama

ruiba168 commented 11 months ago

🐛 Bug

Try to run MLC LLM on an iphone, with codeLlama, Got following error: Check failed: (fload_exec.defined()) is false: TVM runtime cannot find vm_load_executable

It works well for Llama 2.

To Reproduce

Steps to reproduce the behavior:

compile codeLlama with command: python3 -m mlc_llm.build --model codellama_CodeLlama-7b-hf --target iphone --quantization q4f16_1
clean everything under the ios/build
run ./prepare_libs.sh
change the content in ./prepare_params.sh, add codellama_CodeLlama-7b-hf-q4f16_1 in builtin_list section
run ./prepare_params.sh
add codellama_CodeLlama-7b-hf-q4f16_1 in app-config.json in MLCChat app.
run MLCChat app and will get following error.

libc++abi: terminating due to uncaught exception of type tvm::runtime::InternalError: [15:12:48] /Users/I514295/work/project/llm/mlc-llm/cpp/llm_chat.cc:157: InternalError: Check failed: (fload_exec.defined()) is false: TVM runtime cannot find vm_load_executable Stack trace: [bt] (0) 1 ODataML 0x0000000104bc3244 tvm::runtime::detail::LogFatal::Entry::Finalize() + 68 [bt] (1) 2 ODataML 0x0000000104bc3200 tvm::runtime::detail::LogFatal::Entry::Finalize() + 0 [bt] (2) 3 ODataML 0x0000000104bc2274 __clang_call_terminate + 0 [bt] (3) 4 ODataML 0x0000000104bd9c50 mlc::llm::LLMChat::Reload(tvm::runtime::TVMArgValue, tvm::runtime::String, tvm::runtime::String) + 8104 [bt] (4) 5 ODataML 0x0000000104bd7a00 mlc::llm::LLMChatModule::GetFunction(tvm::runtime::String const&, tvm::runtime::ObjectPtr const&)::'lambda'(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue)::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue) const + 680 [bt] (5) 6 ODataML 0x0000000104bd774c tvm::runtime::PackedFuncObj::Extractor<tvm::runtime::PackedFuncSubObj<mlc::llm::LLMChatModule::GetFunction(tvm::runtime::String const&, tvm::runtime::ObjectPtr const&)::'lambda'(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue)>>::Call(tvm::runtime::PackedFuncObj const, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) + 40 [bt] (6) 7 ODataML 0x0000000104ba8334 -[ChatModule reload:modelPath:appConfigJson:] + 520 [bt] (7) 8 ODataML 0x0000000104b8673c $s7ODataML9ChatStateC010mainReloadC033_6907CD1CE42C658DF343ACFA85A59945LL7localID8modelLib0O4Path16estimatedVRAMReq11displayNameySSS2SSiSStFyycfU + 904 [bt] (8) 9 ODataML 0x0000000104bad318 $sIeg_ytIegr_TRTA + 20

Expected behavior

Environment

Platform (e.g. WebGPU/Vulkan/IOS/Android/CUDA):IOS
Operating system (e.g. Ubuntu/Windows/MacOS/...):MacOS
Device (e.g. iPhone 12 Pro, PC+RTX 3090, ...) MyMac(Designed for iPad)
How you installed MLC-LLM (conda, source):conda
How you installed TVM-Unity (pip, source):pip
Python version (e.g. 3.10):3.11
GPU driver version (if applicable): non
CUDA/cuDNN version (if applicable):
TVM Unity Hash Tag (python -c "import tvm; print('\n'.join(f'{k}: {v}' for k, v in tvm.support.libinfo().items()))", applicable if you compile models):USE_NVTX: OFF USE_GTEST: AUTO SUMMARIZE: OFF USE_IOS_RPC: OFF USE_MSC: OFF USE_ETHOSU: CUDA_VERSION: NOT-FOUND USE_LIBBACKTRACE: AUTO DLPACK_PATH: 3rdparty/dlpack/include USE_TENSORRT_CODEGEN: OFF USE_THRUST: OFF USE_TARGET_ONNX: OFF USE_AOT_EXECUTOR: ON BUILD_DUMMY_LIBTVM: OFF USE_CUDNN: OFF USE_TENSORRT_RUNTIME: OFF USE_ARM_COMPUTE_LIB_GRAPH_EXECUTOR: OFF USE_CCACHE: AUTO USE_ARM_COMPUTE_LIB: OFF USE_CPP_RTVM: USE_OPENCL_GTEST: /path/to/opencl/gtest USE_MKL: OFF USE_PT_TVMDSOOP: OFF MLIR_VERSION: NOT-FOUND USE_CLML: OFF USE_STACKVM_RUNTIME: OFF USE_GRAPH_EXECUTOR_CUDA_GRAPH: OFF ROCM_PATH: /opt/rocm USE_DNNL: OFF USE_VITIS_AI: OFF USE_MLIR: OFF USE_RCCL: OFF USE_LLVM: llvm-config --link-static USE_VERILATOR: OFF USE_TF_TVMDSOOP: OFF USE_THREADS: ON USE_MSVC_MT: OFF BACKTRACE_ON_SEGFAULT: OFF USE_GRAPH_EXECUTOR: ON USE_NCCL: OFF USE_ROCBLAS: OFF GIT_COMMIT_HASH: 5e06dab1a3bc691d71230d56d3bb3bb9df1220ce USE_VULKAN: OFF USE_RUST_EXT: OFF USE_CUTLASS: OFF USE_CPP_RPC: OFF USE_HEXAGON: OFF USE_CUSTOM_LOGGING: OFF USE_UMA: OFF USE_FALLBACK_STL_MAP: OFF USE_SORT: ON USE_RTTI: ON GIT_COMMIT_TIME: 2023-11-12 15:22:32 -0500 USE_HEXAGON_SDK: /path/to/sdk USE_BLAS: none USE_ETHOSN: OFF USE_LIBTORCH: OFF USE_RANDOM: ON USE_CUDA: OFF USE_COREML: OFF USE_AMX: OFF BUILD_STATIC_RUNTIME: OFF USE_CMSISNN: OFF USE_KHRONOS_SPIRV: OFF USE_CLML_GRAPH_EXECUTOR: OFF USE_TFLITE: OFF USE_HEXAGON_GTEST: /path/to/hexagon/gtest PICOJSON_PATH: 3rdparty/picojson USE_OPENCL_ENABLE_HOST_PTR: OFF INSTALL_DEV: OFF USE_PROFILER: ON USE_NNPACK: OFF LLVM_VERSION: 15.0.7 USE_OPENCL: OFF COMPILER_RT_PATH: 3rdparty/compiler-rt RANG_PATH: 3rdparty/rang/include USE_SPIRV_KHR_INTEGER_DOT_PRODUCT: OFF USE_OPENMP: OFF USE_BNNS: OFF USE_CUBLAS: OFF USE_METAL: ON USE_MICRO_STANDALONE_RUNTIME: OFF USE_HEXAGON_EXTERNAL_LIBS: OFF USE_ALTERNATIVE_LINKER: AUTO USE_BYODT_POSIT: OFF USE_HEXAGON_RPC: OFF USE_MICRO: OFF DMLC_PATH: 3rdparty/dmlc-core/include INDEX_DEFAULT_I64: ON USE_RELAY_DEBUG: OFF USE_RPC: ON USE_TENSORFLOW_PATH: none TVM_CLML_VERSION: USE_MIOPEN: OFF USE_ROCM: OFF USE_PAPI: OFF USE_CURAND: OFF TVM_CXX_COMPILER_PATH: /Library/Developer/CommandLineTools/usr/bin/c++ HIDE_PRIVATE_SYMBOLS: ON
Any other relevant information:

Additional context

ruiba168 commented 11 months ago

Found renaming the app's identity id will fix the issue.

junrushao commented 11 months ago

Interesting! What is the app's identity id, and why is this causing problems? I've seen a few similar reports and think it will be helpful to the other developers

omkar806 commented 7 months ago

hi I am facing this same issue for running gemma 2b on my mac ? how exactly can change the app's identity ID ? if there are any steps I need to follow ? Also what should be the changed ID ? to run the app successfully ?

mlc-ai / mlc-llm