-
### System Info
GPU: `A10`
Base Image: `FROM nvidia/cuda:12.1.0-runtime-ubuntu22.04`
Tensorrt-llm:
- `0.12.0` : It's working, but I can't use it because of a version mismatch in TRT and trt-llm-back…
-
### 🔖 Feature description
Finally I recently added a swappable base_url for openai client, thus if you configure docsgpt with LLM_NAME=openai
You can run any model you want locally with openai compa…
-
### Your current environment
The output of `python collect_env.py`
```text
Collecting environment information...
PyTorch version: 2.5.1+cpu
Is debug build: False
CUDA used to build PyTorch…
-
### System Info
System:
- CPU Architecture: x86_64
- GPU: NVIDIA H100 - 80GB - CUDA 12.4
- TensorRT-LLM: main branch, commit 535c9cc6730f5ac999e4b1cb621402b58138f819
- Operating System: Ubuntu 22.04…
-
Is it possible to use a local LLM via Ollama. If, what's the setup and what's the requirement for which LLM I can use (guessing it has to use openai api syntax)?
-
### System Info
A100
### Who can help?
_No response_
### Information
- [x] The official example scripts
- [ ] My own modified scripts
### Tasks
- [x] An officially supported task in the `exampl…
-
### Checked other resources
- [X] I searched the Codefuse documentation with the integrated search.
- [X] I used the GitHub search to find a similar question and didn't find it.
- [X] I am sure that …
-
### System Info
- CPU: x86_64
- GPU: A10 (24G)
### Who can help?
_No response_
### Information
- [x] The official example scripts
- [ ] My own modified scripts
### Tasks
- [x] An officially su…
-
### System Info
- CPU: x86_64
- CPU mem: 64GB
- GPU name: V100 SXM2 16GB and Tesla T4 15GB both happens
- Libraries
- TensorRT-LLM commit https://github.com/NVIDIA/TensorRT-LLM/tree/3c46c2794e7f6df48…
-
I attempted to set up using a local LLM in WSL using Ollama. After entering the configuration details for the local model and restarting the app, I encountered the following error. Now, the app won’t …