Closed tstanek390 closed 6 months ago
Maybe you can refer to this example from lamma.cpp community: https://github.com/ggerganov/llama.cpp/discussions/4508
Thx, but I guess I would need GGUF version of the model to run it with LLama.cpp..?
You can convert the model to GGUF follow the instruction of https://github.com/ggerganov/llama.cpp/tree/master/examples/llava. The detailed deployment instruction is in process and will be released soon.
We have updated README.md with the deployment instruction on Android devices with customized llama.cpp
. Have a try!
The inference plan on iPhone is on schedule but not proposed yet. Any ideas?
Thanks for adding this! Are you planning on submitting a PR to llama.cpp, or keeping the forked repo?
We have submitted a PR to llama.cpp
. We want to get the support of open-source communities, since that there are a lot space for optimization of the deployment on mobile devices.
Hi, we are closing this issue due to the inactivity. Hope your question has been resolved. If you have any further concerns, please feel free to re-open it or open a new issue. Thanks!
Hello, thanks for sharing these very promising LLMs. I'd like to run it on iOS, but as I'm a noob at coding and stuff, I don't really know how to start. I'm an healtcare professional and would love to give it a try. Any hints/tips ? Thanks a lot and all the best to the new year! Teodor