-
# Prerequisites
Please answer the following questions for yourself before submitting an issue.
- [ Yes] I am running the latest code. Development is very rapid so there are no tagged versions as…
-
### What happened?
Hi there,
I was trying to build llama.cpp in a project that uses the C++ 23 standard and there are a lot of errors when building the `llama` target with MSVC. The only fix is to d…
-
### What happened?
按照readme中的固件驱动版本，推理时出现报错
### Name and Version
最新版本
### What operating system are you seeing the problem on?
_No response_
### Relevant log output
```shell
llama_new_context_w…
-
Hi,
I am experimenting with gpustack and I noticed when hosting LLMs on CPU only with llama.cpp as backend that only one CPU core is being utilised when inquering such LLM.
Can multicore processi…
-
getting assertion error when starting the synthetic data generation
INFO 2024-11-10 08:00:58,565 instructlab.model.backends.llama_cpp:232: Starting server process, press CTRL+C to shutdown serve…
-
**Is your feature request related to a problem? Please describe.**
Currently I am using Qwen2vl, this is the best vlm model for my project. I hope llama-cpp-python can support this model. I tried to …
-
### What is the issue?
When using the llm benchmark with ollama https://github.com/MinhNgyuen/llm-benchmark , I get around 80 t/s with gemma 2 2b. When asking the same questions to llama.cpp in conve…
-
I downloaded the weights from https://huggingface.co/shuttleai/shuttle-3-diffusion, the program loaded the weights and exit for no error message.
I debugged the program, it seems that the problem i…
-
Quote from readme:
> This project, a Dart binding for llama.cpp, is currently on hold as we await the porting of llama.cpp helpers code to C
Is there a link, potentially to the github issue in the…
-
- it should automatically detect the best device to run on
- We should require 0 manual configuration from the user, by default llama.cpp for example requires specifying the device