-
# Prerequisites
Please answer the following questions for yourself before submitting an issue.
- [x] I am running the latest code. Development is very rapid so there are no tagged versions as of…
-
I am Trying to Load Phi2 GGUF Model via `llama-cpp-python` dependency
The Model I am trying to load is this:
https://huggingface.co/TheBloke/phi-2-GGUF
I am Getting this Error while I am trying …
-
**Is your feature request related to a problem? Please describe.**
It would be nice to intergrate https://llama-cpp-python.readthedocs.io/en/stable/#embeddings because of the speed of default `senten…
-
## Expected Behavior
I expect the binding to be installed successfully when I click "install binding" from the binding zoo
## Current Behavior
Currently it is outputting an error when I install t…
-
When using the latest update with how I've setup my embedded install, I ran into this issue:
```
No module named 'llama_cpp.llama_chat_format'
Traceback (most recent call last):
File "Z:\Anybot…
-
I got Error Code 8: Internal Error (shape calculation overflow) when executing the benchmark using the follow command:
```bash
mpirun -n 8 python benchmark.py -m llama_70b --mode plugin --batch_size…
-
# Prerequisites
Please answer the following questions for yourself before submitting an issue.
- [X] I am running the latest code. Development is very rapid so there are no tagged versions as of…
-
I'm running a finetuning unsloth python code in a colab notebook without subscription and I want to save the gguf in my google drive space directly but the function returns an error. I want to do th…
-
### Title:
Issue with Passing Custom Arguments to `llama_cpp.server` in Docker
#### Issue Description:
Hello `abetlen`,
I've been trying to use your Docker image `ghcr.io/abetlen/llama-cpp-pyt…
-
To apply grammar to chat completion, it looks like the llamafile server is expecting the argument `grammar`: https://github.com/Mozilla-Ocho/llamafile/blob/main/llama.cpp/server/server.cpp#L2551
```
…