-
I might be missing something, but does `llama-cpp-python` release the Python GIL at the moment?
If it doesn't, would releasing the GIL allow multi-threaded execution of GGUF files?
-
# Prerequisites
pip install llama-cpp-python --verbose
# Environment and Context
```
$ python3 --version
Python 3.12.3
$ make --version
GNU Make 3.82
$ g++ --version
gcc (GCC) 11.2.0
```…
-
**Describe the bug**
**To Reproduce**
Steps to reproduce the behavior:
1. Download ilab version 0.21.0
2. Follow the linux installation -
"```
python3 -m venv --upgrade-deps venv
source ven…
-
### Describe what should be investigated or refactored
The LeapfrogAI llama-cpp-python backend has not been updated in a long time, dependencies wise and code-wise. It lacks the necessary code and …
-
# llama_get_logits_ith: invalid logits id -1 error when embedding=True
## Expected Behavior
When using llama-cpp-python with Qwen2 model, the chat completion should work normally regardless of w…
-
Traceback (most recent call last):
File "/data/zhy/models/llama_cpp_python/model_test.py", line 1, in
from llama_cpp import Llama
File "/data/zhy/models/llama_cpp_python/llama_cpp/__init__…
-
# Prerequisites
Please answer the following questions for yourself before submitting an issue.
- [Y] I am running the latest code. Development is very rapid so there are no tagged versions as of…
-
# Prerequisites
Please answer the following questions for yourself before submitting an issue.
- [ ] I am running the latest code. Development is very rapid so there are no tagged versions as of…
-
Right now we call llama.cpp directly, long-term we should go with either llama.cpp directly or llama-cpp-python. Because maintaining two different llama.cpp backends isn't ideal, they will never be in…
-
# Prerequisites
Please answer the following questions for yourself before submitting an issue.
- [x] I am running the latest code. Development is very rapid so there are no tagged versions as of…