a-ghorbani / pocketpal-ai

An app that brings language models directly to your phone.
MIT License
1.14k stars 84 forks source link

Android Crashes: Known Device/Quantization Incompatibilities #107

Open a-ghorbani opened 10 hours ago

a-ghorbani commented 10 hours ago

This issue serves as a repository for tracking Android devices and specific quantizations that lead to crashes.

Please share detailed reports including device model and CPU specs, Android versions, the specific GGUF and its quantization, and stack traces if available.

Phone Model Chipset CPU Configuration App Version Model(s)
Tecno Spark 20 Pro+ MediaTek Helio G99 8-core (2x Cortex-A76 at 2.2 GHz & 6x Cortex-A55 at 2.0 GHz) 1.4.6
OnePlus 12 Qualcomm SM8650-AB Snapdragon 8 Gen 3 (4 nm) 8-core (1x3.3 GHz Cortex-X4 & 3x3.2 GHz Cortex-A720 & 2x3.0 GHz Cortex-A720 & 2x2.3 GHz Cortex-A520) 1.4.6
Motorola Edge 30 Neo Qualcomm SM6375 Snapdragon 695 5G (6 nm) Octa-core (2x2.2 GHz Kryo 660 Gold & 6x1.7 GHz Kryo 660 Silver) 1.5.2 madlad400
Xiaomi Redmi Note 8 Pro Mediatek MT6785V/CC Helio G90T (12 nm) Octa-core (2x2.05 GHz Cortex-A76 & 6x2.0 GHz Cortex-A55) 1.4.6 Whatever model
Samsung Galaxy Note10 Lite Exynos 9810 (10 nm) Octa-core (4x2.7 GHz Mongoose M3 & 4x1.7 GHz Cortex-A55) 1.4.6
Infinix NOTE 30 Mediatek Helio G99 (6 nm) Octa-core (2x2.2 GHz Cortex-A76 & 6x2.0 GHz Cortex-A55) 1.4.6

If you don't know the CPU and chipset of the phone, you can use GSMArena to get the information.

twdragon commented 7 hours ago

I used this T5 model, trying different modes and templates: https://huggingface.co/Felladrin/gguf-q5_k_m-madlad400-3b-mt

Either I had only crashes when run without any template and <2en> prompt header for Spanish text, or unreadable <><>>>< sequences when the default template was used. I think, the llama.cpp kernel should be updated