'get_tensor' error occurs during inference on mobile devices

[wwqq]

I follow the deployment instructions https://github.com/XiaotaoChen/llama.cpp/blob/MobileVLM/examples/llava/MobileVLM-README.md step-by-step. But when I ran the command

sh adb_run.sh

The output file is:

cd /data/local/tmp /data/local/tmp/llava-cli -m /data/local/tmp/ggml-model-q4_k.gguf --mmproj /data/local/tmp/mmproj-model-f16.gguf -t 4 --image /data/local/tmp/demo.jpg -p A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions. USER: <image>
Who is the author of this book? 
Answer the question using a single word or phrase. ASSISTANT:
libc++abi: terminating due to uncaught exception of type std::runtime_error: get_tensor: unable to find tensor mm.model.mlp.1.weight

Aborted 

How to solve this problem？Thanks！

[YangYang-DLUT]

Could you provide the complete log like this? #13

It looks like the models are not loaded successfully. There are two possible reasons:

1. Whether the hf model is completely downloaded. Using git lfs pull is recommended while downloading the model from huggingface. After downloading the model, you can try the inference with python script first if possible.
2. The model is not converted successfully. tensor mm.model.mlp.1.weight lies in mmproj-model-f16.gguf. To convert the model to GGUF, you must have deployed llama.cpp on you PC. You can test the model on you PC first to check if the model is converted successfully then run it on android later. Please make sure all of paths of models are right and the args of command line are in right format.

Later we will upload the GGUF model of MobileVLM somewhere and update a simpler instruction on android deployment.

[wwqq]

git lfs pull

it works. thx