search

mlc-ai / mlc-llm

Universal LLM Deployment Engine with ML Compilation
https://llm.mlc.ai/
Apache License 2.0
18.63k stars 1.51k forks source link

[Bug] Llama-2-7b-chat-hf-q4f16_1-MLC output incorrectly on Andorid(Dimensity 9300) #1498

Closed LingsiDS closed 4 months ago

LingsiDS commented 8 months ago

🐛 Bug

To Reproduce

Steps to reproduce the behavior:

1.download from apk from https://llm.mlc.ai/docs/ [Android tab ] 2.install apk 3.click download button, download base model Llama-2-7b-chat-hf-q4f16_0

Whatever my prompt is, the model outputs a string of "めめめめめめ..."

This problem only occurred on the Dimensity 9300(vivo x100 pro), which worked well on the 8 Gen 2 chip, Dimensity 9200 chip.

tqchen commented 4 months ago

not sure if it is due to out ot memory issue as llama can be too big for lower end phones. closing for now, maybe it is good to try smaller models

BlindDeveloper commented 4 months ago

Good day. @tqchen X100 Pro is a Flagship with 12GB Ram in the minimum version. @LingsiDS Said "This problem only occurred on the Dimensity 9300(vivo x100 pro), which worked well on the 8 Gen 2 chip, Dimensity 9200 chip." can you please compare the previous and new version of the Mlc Chat. It appears that changes have been made that affect the app's working on Android. On device with Mediatek 1080 latest version does not running Gemma 2v, but if i install back Mlc Chat version which downloaded 29.03.2024 Gemma 2b working. @LingsiDS is the amount of operative memory the same for both devices with mediatek?."