-
To get this to work, first you have to get an external AMD GPU working on Pi OS. The most up-to-date instructions are currently on my website: [Get an AMD Radeon 6000/7000-series GPU running on Pi 5](…
-
## Goal
- Jan supports most llama.cpp params
## Tasklist
**Cortex**
- [x] https://github.com/janhq/cortex.cpp/issues/1151
**Jan**
- [ ] Update Right Sidebar UX for Jan
- [ ] Enable Jan's API serv…
-
### Git commit
5a8987793f3e7c1fbfa6806bfcd17d578071b6c9
### Operating systems
Windows
### GGML backends
CUDA
### Problem description & steps to reproduce
Whilst trying to improve performance o…
aendk updated
19 hours ago
-
Hi there,
First off, thanks for the hard work creating this package.
I am currently having some issues getting the package to run on iOS. I am currently loading both dynamic libraries `libggml.d…
-
Llama.cpp has recently added support for the [DRY](https://github.com/oobabooga/text-generation-webui/pull/5677) and [XTC](https://github.com/oobabooga/text-generation-webui/pull/6335) sampling algori…
p-e-w updated
3 weeks ago
-
When will llama.cpp and the Mistral Nemo PR be updated to the latest version?
-
### What happened?
When forcing llama.cpp to use "GPU + CUDA + VRAM + shared memory (UMA)", we noticed:
- High CPU load (even when only GPU should be used)
- Worse performance than using "CPU + RAM…
-
inference太慢了。有没llama.cpp支持？
-
# Current Behavior
I run the following:
CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python --verbose
an error occured:
ERROR: Failed building wheel for llama-cpp-python
# Environment …
-
Hi @lamikr,
I built rocm_sdk_builder on a freshly installed Ubuntu 24.04.1. It took 5 hours, 120GB of storage and many hours of fixing small issues during building the repo (reference: https://gith…