-
### Discussed in https://github.com/ggerganov/llama.cpp/discussions/9043
Originally posted by **julmb** August 15, 2024
Let's say I want to use llama.cpp as a shared library to build a service…
-
## Describe the bug
llama-toolchain python package
### Minimal reproducible example
Installing with ```pip install llama-toolchain```
Running llama or any other parameter results in a crash wit…
-
I apologize if this is not the appropriate place for questions, concerns, or suggestions regarding the project.
One of the major challenges with AI is how quickly things progress, and understanding…
-
### Bug Description
I keep getting a pydantic error when trying to retrieve from the vector DB from an existing collection. I can see that the points are retrieved from the Qdrant vectorDB.
```bas…
-
Hi, any chance you can upload modeling file for Llama 7B equivalent DiJiang?
-
### What happened?
This started as a problem with Ooba, but I'm seeing the same issue with KoboldCPP and llama.cpp. I updated Ooba the other day, after maybe a week or two of not doing so. While it …
-
### Bug Description
The callback manager set directly on the LLM is removed when using `VectorStoreIndex.as_chat_engine`.
This is problematic because it makes token counting impossible.
I belie…
-
### What happened?
*Title**: Feature Request: Add option for explicit Metal device selection on macOS
---
**Description**:
Hi,
I'm using `llama.cpp` on a macOS system with multiple GPUs, …
-
Hi,
I am following the article at https://learn.arm.com/learning-paths/servers-and-cloud-computing/pytorch-llama/pytorch-llama/
but at step
```
python torchchat.py export llama3.1 --output-dso-p…
-
Related to #15 #17
I'm updating aub.ai to make use the latest version of llama.cpp. This update has deprecated the `llama_eval` function in favor of `llama_decode`, which now requires the use of `l…