[Bug] some questions about build Android app with mlc

[ecccccsgo]

🐛 Bug

I hope to build Android app on SP 8+ phone, and hope to use it further in SP 8155 chip and other chips (used in car). Here are my operations:

1. complete the environment described in the docs.
2. download qwen2.5-0.5B from qwen repo.
3. convert weights: mlc_llm convert_weight dist/models/Qwen2.5-0.5B-Instruct --quantization q4f16_1 -o dist/models/Qwen2.5-0.5B-Instruct_MLC output includes: [2024-10-17 17:43:01] INFO auto_device.py:88: Not found device: cuda:0 [2024-10-17 17:43:02] INFO auto_device.py:88: Not found device: rocm:0 [2024-10-17 17:43:04] INFO auto_device.py:88: Not found device: metal:0 [2024-10-17 17:43:06] INFO auto_device.py:79: Found device: vulkan:0 [2024-10-17 17:43:07] INFO auto_device.py:88: Not found device: opencl:0 [2024-10-17 17:43:07] INFO auto_device.py:35: Using device: vulkan:0 ...... Saved to directory: .....
4. generate config:

   
             --quantization q4f16_1 --conv-template qwen2 \
             -o dist/models/Qwen2.5-0.5B-Instruct_MLC/```

5. change the file in mlc-llm/android/MLCChat/mlc-package-config.json:

     {
               "device": "android",
               "model_list": [
                   {
                       "model": "../../dist/models/Qwen2.5-0.5B-Instruct_MLC",
                       "estimated_vram_bytes": 738740000,
                       "model_id": "Qwen2.5-0.5B-Instruct-q4f16_1-MLC",
                       "bundle_weight": true
                   }
               ]
           }

6. run mlc_llm package to compile model. and build android project.
7. install app, move MLC weights to Android\data\ai.mlc.mlcchat\files.

On SP 8+ phone it works fine.

on SP 8155 it crash:

10-15 19:18:12.385  8428  8461 E AndroidRuntime: FATAL EXCEPTION: Thread-4
10-15 19:18:12.385  8428  8461 E AndroidRuntime: Process: ai.mlc.mlcchat, PID: 8428
10-15 19:18:12.385  8428  8461 E AndroidRuntime: org.apache.tvm.Base$TVMError: InternalError: Check failed: (m_libHandler != nullptr) is false: Error! Cannot open libOpenCL!
10-15 19:18:12.385  8428  8461 E AndroidRuntime: Stack trace:
10-15 19:18:12.385  8428  8461 E AndroidRuntime:   File "/home/csg/mlc-llm/android/mlc4j/../../3rdparty/tvm/src/runtime/opencl/opencl_wrapper/opencl_wrapper.cc", line 128
10-15 19:18:12.385  8428  8461 E AndroidRuntime: 
10-15 19:18:12.385  8428  8461 E AndroidRuntime:    at org.apache.tvm.Base.checkCall(Base.java:173)
10-15 19:18:12.385  8428  8461 E AndroidRuntime:    at org.apache.tvm.Function.invoke(Function.java:130)
10-15 19:18:12.385  8428  8461 E AndroidRuntime:    at ai.mlc.mlcllm.JSONFFIEngine.runBackgroundLoop(JSONFFIEngine.java:64)
10-15 19:18:12.385  8428  8461 E AndroidRuntime:    at ai.mlc.mlcllm.MLCEngine$backgroundWorker$1.invoke(MLCEngine.kt:42)
10-15 19:18:12.385  8428  8461 E AndroidRuntime:    at ai.mlc.mlcllm.MLCEngine$backgroundWorker$1.invoke(MLCEngine.kt:40)
10-15 19:18:12.385  8428  8461 E AndroidRuntime:    at ai.mlc.mlcllm.BackgroundWorker$start$1.invoke(MLCEngine.kt:19)
10-15 19:18:12.385  8428  8461 E AndroidRuntime:    at ai.mlc.mlcllm.BackgroundWorker$start$1.invoke(MLCEngine.kt:18)
10-15 19:18:12.385  8428  8461 E AndroidRuntime:    at kotlin.concurrent.ThreadsKt$thread$thread$1.run(Thread.kt:30)

i hope to know is my environment in compile model wrong? Found device: vulkan:0 and Not found device: opencl:0.

Am i need to use opencl to compile the model?

or my problem in other places?

please help me!!

[MasterJH5574]

Hi @ecccccsgo thank you for the question. It doesn't require your host machine to have OpenCL when you compile the model, but MLC right now requires your device (i.e., your "SP 8+ phone") to have the OpenCL driver.

From the error message you shared,

10-15 19:18:12.385 8428 8461 E AndroidRuntime: FATAL EXCEPTION: Thread-4
10-15 19:18:12.385 8428 8461 E AndroidRuntime: Process: ai.mlc.mlcchat, PID: 8428
10-15 19:18:12.385 8428 8461 E AndroidRuntime: org.apache.tvm.Base$TVMError: InternalError: Check failed: (m_libHandler != nullptr) is false: Error! Cannot open libOpenCL!

It looks to me that libOpenCL does not exist on your device.

So would you mind first confirming whether your phone comes with the OpenCL driver? Please let us know if there are further updates. Thanks.

[ecccccsgo]

@MasterJH5574 I get it! Thank you~ I will work to it.

I also hope to know that how to get "estimated_vram_bytes" in mlc-llm/android/MLCChat/mlc-package-config.json, did i need to run on my phone?

[MasterJH5574]

@ecccccsgo Thanks for the question. I think for now it doesn't have effects in the Android app so you can set an arbitrary one I guess.