[Bug] Download model config failed: no protocol

LCmoon0126 commented 3 months ago

🐛 Bug

To Reproduce

Steps to reproduce the behavior:

I convert model weights and package libraries and weights on my macOS
So my mlc-package-config.json in ./android/MLCChat is like this and the 'model' is a local path { "device": "android", "model_list": [ { "model": "/Users/licheng/Desktop/AI/models/gemma2/gemma-2-2b/gemma-2-2b-q4f16_1-MLC", "model_id": "gemma-2-2b-it-q4f16_1-MLC", "estimated_vram_bytes": 3000000000, "bundle_weight": true } ] }
Then I run the Anrdoid and it shows Download model config failed: no protocol: /Users/licheng/Desktop/AI/models/gemma2/gemma-2-2b/gemma-2-2b-q4f16_1-MLC/resolve/main/mlc-chat-config.json

Expected behavior

Environment

Platform (e.g. WebGPU/Vulkan/IOS/Android/CUDA): Android
Operating system (e.g. Ubuntu/Windows/MacOS/...): MacOS
Device (e.g. iPhone 12 Pro, PC+RTX 3090, ...): Android
How you installed MLC-LLM (conda, source):
How you installed TVM-Unity (pip, source):
Python version (e.g. 3.10):
GPU driver version (if applicable):
CUDA/cuDNN version (if applicable):
TVM Unity Hash Tag (python -c "import tvm; print('\n'.join(f'{k}: {v}' for k, v in tvm.support.libinfo().items()))", applicable if you compile models):
Any other relevant information:

Additional context

I see in the doc about ackage_libraries_and_weights, it says in mlc-package-config.json the 'model':

model (Required) The path to the MLC-converted model to be built into the app.

Usually it is a Hugging Face URL (e.g., "model": "HF://mlc-ai/phi-2-q4f16_1-MLC"`) that contains the pre-converted model weights. For iOS, it can also be a path to a local model directory which contains converted model weights (e.g., "model": "../dist/gemma-2b-q4f16_1"). Please check out Convert Model Weights if you want to build local model into the app.

Is that means I can't use a local convert model in Android? Thank you very much

tqchen commented 2 months ago

Because android have APK size limit, the model is usually larger than that cap. So we will need to store the model online and download it after APK get started.

There are some advanced option to do local bundling per https://llm.mlc.ai/docs/deploy/android.html#bundle-model-weights, which may work for local source assuming you copy the files correctly. If not it should be possible to use that path and update the code a bit to support local bundling

LCmoon0126 commented 2 months ago

Because android have APK size limit, the model is usually larger than that cap. So we will need to store the model online and download it after APK get started.

There are some advanced option to do local bundling per https://llm.mlc.ai/docs/deploy/android.html#bundle-model-weights, which may work for local source assuming you copy the files correctly. If not it should be possible to use that path and update the code a bit to support local bundling

Got it, thank you very much

mlc-ai / mlc-llm