workbench for learing&practising AI tech in real scenario on Android device, powered by GGML(Georgi Gerganov Machine Learning) and NCNN(Tencent NCNN) and FFmpeg
Apache License 2.0
117
stars
19
forks
source link
[llama.cpp] AI answer does not stop automatically when inference is launched on Android phone #116
here is a screenshot to demonstrate this issue(the input question is: where is China's capital city?):
same issue here with following models on Xiaomi 14 (Xiaomi 14 is available since Oct 2023, Xiaomi 14 contains a very very very powerful mobile SoC ------ Qualcomm SM8650-AB Snapdragon 8 Gen 3 (4 nm) ------ and was used for personal device-side AI PoC development activity).
CPU-only reference on Xiaomi 14.
here is a screenshot to demonstrate this issue(the input question is: where is China's capital city?):
same issue here with following models on Xiaomi 14 (Xiaomi 14 is available since Oct 2023, Xiaomi 14 contains a very very very powerful mobile SoC ------ Qualcomm SM8650-AB Snapdragon 8 Gen 3 (4 nm) ------ and was used for personal device-side AI PoC development activity).
original similar issue report could be found at upstream llama.cpp:
https://github.com/ggerganov/llama.cpp/issues/5034
https://github.com/ggerganov/llama.cpp/issues/3969