-
### Before submitting your bug report
- [X] I believe this is a bug. I'll try to join the [Continue Discord](https://discord.gg/NWtdYexhMs) for questions
- [X] I'm not able to find an [open issue]…
-
### What happened?
First of all - I am not sure whether this is a issue but, I could not find any answers anywhere.
I am trying to run a Llama3 8B model on a Raspberry pi5. I am using llama.cpp f…
-
### Summary:
When testing the latest version of llama-cpp-python (0.1.64) alongside [the corresponding commit of llama.cpp](https://github.com/ggerganov/llama.cpp/tree/8596af427722775f0df4a7c90b9af06…
-
I've been using llama.cpp for quite a while (M1 Mac). Is there a way I can get ai_voicetalk_local.py to point to that installation instead of reinstalling it here? Sorry, newbie question...
-
On adding llama_cpp-rs to my Cargo.toml, llama.cpp seems to be locked to an older version. I'm trying to use Phi-3 128k in a project and I'm unable to because the [PR that was merged into llama.cpp](h…
-
### Why is it worth to add this package?
Port of Facebook's LLaMA model in C/C++
### Home page URL
https://github.com/ggerganov/llama.cpp
### Source code URL
https://github.com/ggerganov/llama.cp…
-
INB4 This is **not about setting Top-P to 1.0** which causes the same output every time for every same prompt, documented here: https://github.com/abetlen/llama-cpp-python/issues/1797
When loading …
-
### Cortex version
Jan v0.5.7 | Cortex v-
### Describe the Bug
https://discord.com/channels/1107178041848909847/1300098068980568095
A known issue exists with the llama.cpp engine’s handling of s…
-
Will you consider supporting the llama.cpp server API for inference?
-
**Qwen2**
warning: not compiled with GPU offload support, --n-gpu-layers option will be ignored
warning: see main README.md for information on enabling GPU BLAS support
Log start
main: build = 2…