-
(yuna2) (base) adm@Adms-MacBook-Pro yuna-ai % python index.py
ggml_metal_init: error: Error Domain=MTLLibraryErrorDomain Code=3 "program_source:3:10: fatal error: 'ggml-common.h' file not found
#inc…
-
Couqi Engine takes brakes mid sentence to load. IT takes sometimes between words or even in the middle of say the word. I tried to adjust setting but nothing works. I use i7 10th and RTX3060 computer.
-
# Expected Behavior
I want llama-cpp-python to be able to load GGUF models with GPU inside docker. It works properly while installing llama-cpp-python on interactive mode but not inside the dockerf…
-
### What happened?
The model loading fails (for example when loading https://huggingface.co/unsloth/Llama-3.2-1B-Instruct-GGUF/tree/main Llama-3.2-1B-Instruct-Q4_K_M.gguf) because of One Definition…
-
I set up Vicuna on a local server, but when calling the API, an error occurred.
```json
{
"detail": [{
"loc": ["body", "max_tokens"],
"msg": "none is not an allowed value",
"type": "type_…
-
Hey, thank you so much for the great model and this repo!
Would you be willing to add support for this chat format to llama-cpp-python, so that we can use function calling (and JSON mode) with thei…
-
Can you please add local LLM support, please?
Ollama support will be nice too.
Thank you.
-
This is how I am loading the model using Python, but it uses only the CPU:
`
Llama(model_path="./functionary-7b-v2.q4_0.gguf", n_ctx=4096, n_gpu_layers=50)
`
I have also tried to re-install ll…
-
## Issues
I am trying to install the lastest version of llama-cpp-python in my windows 11 with RTX-3090ti(24G). I have successfully installed llama-cpp-python=0.1.87 (can't exactly remember) months a…
-
I'm attempting to install llama-cpp-python with GPU enabled on my Windows 11 work computer but am encountering some issues at the very end. The error that I am receiving is "CMake Error: Error requi…