-
Hi there! I'm having an issue where it seems like the LLM isn't able to actually *see* the output from CGRAG. I've tested this with a few different LLMs (although they are all through the same provide…
-
### What happened?
When attempting to quantize [Qwen2 7B instruct](https://huggingface.co/Qwen/Qwen2-7B-Instruct) to IQ2_XS I get the following assert:
```
GGML_ASSERT: ggml-quants.c:12083: gri…
-
Hi,
First and foremost, I want to express my gratitude for the idea behind this project and the way you keep up with the development.
3 days ago I created a small issue regarding [PHI-3 demo her…
-
A.I says:
I encountered an error while trying to use the tool. This was the error: SerperDevTool._run() missing 1 required positional argument: 'search_query'.
Tool Search the internet accepts thes…
-
Opening a new issue (see https://github.com/ollama/ollama/pull/2195) to track support for integrated GPUs. I have a AMD 5800U CPU with integrated graphics. As far as i did research ROCR lately does su…
-
### What is the issue?
I'm using `llama3:70b` through the OpenAI-compatible endpoint. When generating, I am getting outputs like this:
```
Please provide the output of the above command. …
moyix updated
5 months ago
-
Officially ROCm no longer supports these cards, but it looks like other projects have found workarounds. Let's explore if that's possible. Best case, built-in to our binaries. Fall-back if that's n…
-
### System Info
- TensorRT-LLM v0.8.0 (pinned to release commit)
- Nvidia A100
- Mistral-7B-Instruct-v0.2
- Using the CPP runner
- Installed with `pip install tensorrt_llm==0.8.0 --extra-index-ur…
iibw updated
2 months ago
-
On Ubuntu 24.04 I get the following output and error after running `MODEL=phi3 docker-compose up` and then typing a message in the web UI and hitting enter. No response shows up in the UI.
```
(ba…
-
Using streamText with Ollama provider yields AI_JSONParseError.
It seems like everything's working, except it tries to `JSON.parse()` json snippit before they're fully read into the buffer.
Any…