search

mlc-ai / mlc-llm

Universal LLM Deployment Engine with ML Compilation
https://llm.mlc.ai/
Apache License 2.0
19.09k stars 1.56k forks source link

[Bug] TVMruntime cannot find vm_load_executable only for phi-2 on Android #1692

Closed qc903113684 closed 9 months ago

qc903113684 commented 9 months ago

🐛 Bug

I compile phi-2 and Mistral-7B for Android. And Mistral-7B worked but phi-2 get error. 1c595a8e4872d633237ca0f8f43478d

To Reproduce

Steps to reproduce the behavior:

Follow document to compile phi-2 and Mistral and build Android app (https://llm.mlc.ai/docs/deploy/android.html)

Expected behavior

phi-2 work success

Environment

Additional context

I'd like to know if this error means it's not currently supported for phi-2 on Android or if there's something wrong with my compilation process。

Kartik14 commented 9 months ago

Copying from Discord

I think that the app is not able to locate the model library correctly? It might be if the model_lib is not set correctly in app-config.json while building the apk (Refer https://llm.mlc.ai/docs/deploy/android.html#create-android-project-using-compiled-models for more details). Can you share your app-config.json file that you are using to compile the model lib?

qc903113684 commented 9 months ago

Thanks, I fixed this error.

hmzdtz commented 8 months ago

@qc903113684 @Kartik14 May I ask how to fix it? I didn't explicitly use --system-lib-prefix, and all with default value. All other steps and documents are completely consistent, and no exceptions were thrown throughout the process.

This my app-config.json:

{
  "model_list": [
    {
      "model_url": "https://huggingface.co/mlc-ai/Llama-2-7b-chat-hf-q4f16_1-MLC/",
      "model_lib": "llama_q4f16_1",
      "estimated_vram_bytes": 4348727787,
      "model_id": "Llama-2-7b-chat-hf-q4f16_1"
    },
    {
      "model_url": "https://huggingface.co/mlc-ai/RedPajama-INCITE-Chat-3B-v1-q4f16_1-MLC/",
      "model_lib": "gpt_neox_q4f16_1",
      "estimated_vram_bytes": 1948348579,
      "model_id": "RedPajama-INCITE-Chat-3B-v1-q4f16_1"
    }
  ],
  "model_lib_path_for_prepare_libs": {
    "llama_q4f16_1": "libs/Llama-2-7b-chat-hf-q4f16_1-android.tar",
    "gpt_neox_q4f16_1": "libs/RedPajama-INCITE-Chat-3B-v1-q4f16_1-android.tar"
  }
}