search

a-ghorbani / PocketPal-feedback

181 stars 4 forks source link

Crash when loading Llama 3.2 #25

Open dsvf opened 1 week ago

dsvf commented 1 week ago

Not much to add... Download seems to work fine, but when I try to load the model, the app just closes. Downloading and loading e.g. Qwen works fine. Also, the Llama 3.2 3B Q4 gguf from HF works well with ChatterUI, so my device (Samsung Galaxy S23) is in principle capable.

I have adb ready if you want me to test a debug build.

a-ghorbani commented 1 week ago

My guess is that the app might not be detecting the correct architecture. the app is relying on this code to load the right lib: https://github.com/mybigday/llama.rn/blob/f35545b92241d8cc9ee2b22b5ca4c02b395db9bb/android/src/main/java/com/rnllama/LlamaContext.java#L249-L280

Can you please share adb logcat | grep -i rnllama and adb logcat | grep -i "fatal"

dsvf commented 1 week ago 
09-26 23:40:56.583 22955 23001 D RNLlamaContext: CPU features: Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 asimdfhm dit uscat ilrcpc flagm ssbs sb paca pacg dcpodp flagm2 frint i8mm bf16 bti
09-26 23:40:56.583 22955 23001 D RNLlamaContext: Loading librnllama_v8_4_fp16_dotprod.so
09-26 23:40:56.685 22955 23001 I RNLLAMA_ANDROID_JNI: [RNLlama] is_model_loaded false
09-26 23:40:57.089 23062 23062 F DEBUG   :       #00 pc 000000000019000c  /data/app/~~2tpNUTiQeWDVF25W55iPlA==/com.pocketpalai-40LK3F8n7t6T_BmTb_tQgg==/split_config.arm64_v8a.apk!librnllama_v8_4_fp16_dotprod.so (offset 0x12da000) (Java_com_rnllama_LlamaContext_loadModelDetails+104) (BuildId: cc19b1db93c0b29ad31f7f4c8865a6ad4ffa0a30)
09-26 23:40:57.089 23062 23062 F DEBUG   :       #03 pc 00000000004863ee  /data/app/~~2tpNUTiQeWDVF25W55iPlA==/com.pocketpalai-40LK3F8n7t6T_BmTb_tQgg==/base.apk (com.rnllama.LlamaContext.<init>+446)
09-26 23:40:57.089 23062 23062 F DEBUG   :       #05 pc 00000000004868ce  /data/app/~~2tpNUTiQeWDVF25W55iPlA==/com.pocketpalai-40LK3F8n7t6T_BmTb_tQgg==/base.apk (com.rnllama.RNLlama$1.doInBackground+46)
09-26 23:40:57.089 23062 23062 F DEBUG   :       #07 pc 0000000000486980  /data/app/~~2tpNUTiQeWDVF25W55iPlA==/com.pocketpalai-40LK3F8n7t6T_BmTb_tQgg==/base.apk (com.rnllama.RNLlama$1.doInBackground+4)
09-26 23:40:57.136 22594 22594 I Dc.IafdService: handleAppError, pkgName : com.pocketpalai, userId : 0, errorType : 0, repeat : true component : com.pocketpalai-40LK3F8n7t6T_BmTb_tQgg==/split_config.arm64_v8a.apk!librnllama_v8_4_fp16_dotprod.so (offset 0x12da000) (Java_com_rnllama_LlamaContext_loadModelDetails+104) (BuildId: cc19b1db93c0b29ad31f7f4c8865a6ad4ffa0a30)errorStack:       #00 pc 000000000019000c  /data/app/~~2tpNUTiQeWDVF25W55iPlA==/com.pocketpalai-40LK3F8n7t6T_BmTb_tQgg==/split_config.arm64_v8a.apk!librnllama_v8_4_fp16_dotprod.so (offset 0x12da000) (Java_com_rnllama_LlamaContext_loadModelDetails+104) (BuildId: cc19b1db93c0b29ad31f7f4c8865a6ad4ffa0a30)

and

09-26 23:42:07.460 23218 23283 F libc : Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x910 in tid 23283 (AsyncTask #2), pid 23218 (com.pocketpalai)

dsvf commented 1 week ago

Okay, what works is Llama-3.2-3b-instruct-q4_k_m.gguf when downloaded and included as a local model.

LoganTann commented 5 days ago

Had the same issue with Llama 3.2 3B and Quen 2 1.5B on my pixel 7 phone. The app just crashes when I try to load the model.

If I have time I'll also check the logs

ImadSaddik commented 7 hours ago

I have Vivo X50 with 12 GB of RAM and I am also experiencing the same issue with Llama 3.2 1B model.