-
## Implementation
- [ ] Look into LiteLLM (OpenAI, Anthropic, OpenAI Azure support) + Ollama
- [ ] Add LLM connection setup to user settings + global default
- [ ] Replace OpenAI setup popup by LLM…
-
### System Info
- CPU architecture: x86_64
- CPU/Host memory size (if known): 40G
- GPU name :RTX 3060-6G
- TensorRT-LLM branch or tag TensorRT-LLM commit: TensorRT-LLM version: 0.14.0.dev2024092400
…
-
### Your current environment
### Model Input Dumps
_No response_
### 🐛 Describe the bug
```text
The channel dimension is ambiguous. Got image shape (1, 1, 3). Assuming channels are the firs…
-
## What
Add Ollama in as an LLM option.
## Why
Many people run local LLMs, and then knowledge table can be run without an internet connection.
## Implementation guidance
The FastAPI b…
-
https://ollama.com/download
-
I love the fact that we can now use an Open Ai api key within the new Ai Chat feature but also think that it would be great if you could add support for using an LLM hosted locally for this also. I li…
-
### Privileged issue
- [X] I am a LangChain maintainer, or was asked directly by a LangChain maintainer to create an issue here.
### Issue Content
Context: Open (meaning weights-available her…
-
kou@SERVER04:/data/ghj/TensorRT-LLM$ make -C docker build
make: 进入目录“/data/ghj/TensorRT-LLM/docker”
Building docker image: tensorrt_llm/devel:latest
DOCKER_BUILDKIT=1 docker build --pull \
--…
-
* I am following the example of BART LoRA at [here](https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/enc_dec#run-bart-with-lora). I cannot convert the engines without flag `--context_fmha dis…
-
Not really an issue per say. Is there any way to get this to use a local LLM server like Ollama or LM Studio instead of having to pay OpenAI fees? Other than that, great work.