workbench for learing&practising AI tech in real scenario on Android device, powered by GGML(Georgi Gerganov Machine Learning) and NCNN(Tencent NCNN) and FFmpeg
Apache License 2.0
96
stars
17
forks
source link
ggml-jni: update latest source code of ggml/whispercpp/llamacpp from upstream #201
update latest source code of ggml/whispercpp/llamacpp from upstream( for purpose of trying to make a GPT-4V style multimodal LLM MiniCPM-V works fine on Xiaomi 14)
refine ggml-jni/project accordingly for purpose of update/merge latest source code of ggml/whispercpp/llamacpp more easily&quickly in the next time to reduce redundant meaningless work load
Status
update source code of ggml/whispercpp/llamacpp from upstream and refine ggml-jni/project accordingly
There are many issues during update source code from upstream ggml due to the huge changes in the latest source of ggml/whispercpp/llamacpp from upstream.
the following is a example which I can't believe this because this change cause crash in ggml-qnn.cpp:
some issues already fixed and I'm working on figure out why ggml-qnn.cpp does not works fine with latest source code from upstream llamacpp as expected(ggml-qnn.cpp works fine as expected with llamacpp in branch kantv-1.3.7).
it seems the maintainers of upstream llamacpp are working on backward inference(for model training?) in upstream ggml. respect!
validation on Xiaomi14
ASR(using whisper.cpp) works fine/well as expected with QNN backend;
llm inference with QNN backend does't work fine as expected with QNN backend(ggml-qnn.cpp works fine as expected with llamacpp in branch kantv-1.3.7)
minicpm-v inference crash on Xiaomi14(minicpm-v inference works fine on Linux(Ubuntu 20.04))
any other test cases works fine as expected or keep same behavior with branch kantv-1.3.7 on Xiaomi14
Purpose
This PR is intend to
update latest source code of ggml/whispercpp/llamacpp from upstream( for purpose of trying to make a GPT-4V style multimodal LLM MiniCPM-V works fine on Xiaomi 14)
refine ggml-jni/project accordingly for purpose of update/merge latest source code of ggml/whispercpp/llamacpp more easily&quickly in the next time to reduce redundant meaningless work load
Status
There are many issues during update source code from upstream ggml due to the huge changes in the latest source of ggml/whispercpp/llamacpp from upstream.
the following is a example which I can't believe this because this change cause crash in ggml-qnn.cpp:
some issues already fixed and I'm working on figure out why ggml-qnn.cpp does not works fine with latest source code from upstream llamacpp as expected(ggml-qnn.cpp works fine as expected with llamacpp in branch kantv-1.3.7).
it seems the maintainers of upstream llamacpp are working on backward inference(for model training?) in upstream ggml. respect!
close and move to #203