search

UbiquitousLearning / mllm

Fast Multimodal LLM on Mobile Devices
https://ubiquitouslearning.github.io/mllm_website
MIT License
576 stars 63 forks source link

Vocab file for a model to use NPU. #207

Open kjh-icnl opened 19 hours ago

kjh-icnl commented 19 hours ago

Most of all, thank you for your team's awesome trial and application.

By the way, I'm trying to run Qwen-1.5 1.8B with NPU on my mobile phone with an application (ChatBotApp). With your homepage documents, it says Fuyu model has a vocab_uni.mllm for vocabulary and LLaMA model has a vocab.mllm for vocabulary.

Then, which file does Qwen-1.5 1.8B model have for a vocab? Or is it possible to run it on the app without a vocab file? Thank you for your answering in advance.

yirongjie commented 19 hours ago

Thank you for your attention to our project. The vocab file required by the app is marked here .

kjh-icnl commented 18 hours ago

I got you. Thank you.

This is another question though. I read a closed issue about running NPU on S24. That says tensor io almost 8GB and totally almost 16GB.

Galaxy has an extension functionality for RAM, namely RAM Plus. Have you run this app with RAM plus?And which phones does your team test on?

yirongjie commented 18 hours ago

We only tested two mobile phones using the Qualcomm Snapdragon 8 Gen 3 SoC. Their RAM is 16GB (Xiaomi 14) and 24GB (Redmi K70 Pro). In addition, our QNN-based driver for the NPU only supports Qualcomm's SOC.