How to implement a single whispercpp with qnn on Qualcomm SoC?

[Weisonorz]

Thanks for your work. I am new on using Qualcomm SoC systems. Just curious how can we implement whisper or llama cpp separately from this project?

[zhouwg]

I'm sorry I don't know what's the exact meaning you said. the whisper.cpp and llama.cpp can works both well on Qualcomm mobile SoC equipped Android phone.

1. comment line 15&16 in CMakeLists.txt to remove the entire NCNN native source codes, and then remove the ncnn-jni codes accordingly(otherwise the APK will complain JNI error and terminate abnormally)
2. modify the CMakeLists.txt accordingly as your need, and then you can run whisper.cpp or llama.cpp separately.

updated on 06/14/2024: I think I know what you said. there is a simple/direct approach I provided in this project as following,

pls refer to:

https://github.com/zhouwg/kantv/blob/master/core/ggml/llamacpp/tests/ggml-qnn-ut-in-kantv/test-qnn-ops.cpp#L1561

this is a simple UT of whisper inference in Android command line mode which I use it for verify ggml qnn backend on Qualcomm mobile SoC equipped Android phone, but it also works well in Android APK(you can verify in the Android APK which built from the source code of this project).

for real AI application/scenario in commercial Android APK, pls refer to realtime AI subtitle powered by whisper.cpp for online TV in this project.

[zhouwg]

I'd like to close this opening issue accordingly. of course, you can re-open it in the future as your need.thanks for your understanding.