-
llama-cpp-python, Fails to build wheels, Any help or guideance on Windows Server?
```
PS C:\development\localGPT> pip install llama-cpp-python
...
note: This error originates from a subprocess…
-
Hi,
I am running Windows 11, Python 3.11.9, and comfyui in a venv environment.
I tried installing the latest llama-cpp-python for Cuda 1.24 in the below manner and received a string of errors. Can a…
-
After fine tuning the GPT-OCR 2.0 model in Vietnamese language, I obtained a model deployed on the GPU. Can you tell me where to start with the steps to implement it on CPU? I thank you very much.
-
# Prerequisites
Please answer the following questions for yourself before submitting an issue.
- [x] I am running the latest code. Development is very rapid so there are no tagged versions as of…
-
Running the official Qwen 72B GGUF gives no output with tokens bigger then ~2000 tokens, while running the same prompt through llama.cpp with the same settings directly does give output.
I tried re…
-
### Command:
**llama stack run Llama3.2-11B-Vision-Instruct --port 5000**
**Output:**
```
Using config `/Users/mac/.llama/builds/conda/Llama3.2-11B-Vision-Instruct-run.yaml`
Resolved 4 prov…
-
Hi,
I am running some accuracy tests with quantized models. I run the following commands but it takes a long time to run a small test and apparently NVIDIA GPU is not utilized while its available:
…
Znbne updated
1 month ago
-
I am trying to run `./run.sh --model 7b`, i get the following error log ending with `llama-gpt-llama-gpt-api-1 exited with code 1`
`llama-gpt-llama-gpt-api-1 | /usr/local/lib/python3.11/site-packa…
-
### Feature request
I would like to request [llama.cpp](https://github.com/ggerganov/llama.cpp) as a new model backend in the transformers library.
### Motivation
llama.cpp offers:
1) Exce…
-
There's no supported model list or requirement in the doc.
e.g.
I downloaded instructlab/granite-7b-lab but can not serve with it.
```
(dlipy3) [root@llm opt]# ls chu/gran…