-
Hi, thanks for building and opening Savvy!
Is there any way I can configure it to use a locally-running LLM? With OpenAI-compatible API or otherwise.
Thanks!
-
My code：
```
import typing as t
import asyncio
from typing import List
from datasets import load_dataset, load_from_disk
from ragas.metrics import faithfulness, context_recall, context_precisi…
-
- [LMStudio](https://lmstudio.ai/)
- [Continue](https://continue.dev/docs/intro)
-
### System Info
GPU: `A10`
Base Image: `FROM nvidia/cuda:12.1.0-runtime-ubuntu22.04`
Tensorrt-llm:
- `0.12.0` : It's working, but I can't use it because of a version mismatch in TRT and trt-llm-back…
-
### 🔖 Feature description
Finally I recently added a swappable base_url for openai client, thus if you configure docsgpt with LLM_NAME=openai
You can run any model you want locally with openai compa…
-
Is it possible to use a local LLM via Ollama. If, what's the setup and what's the requirement for which LLM I can use (guessing it has to use openai api syntax)?
-
### System Info
- CPU: x86_64
- GPU: A10 (24G)
### Who can help?
_No response_
### Information
- [x] The official example scripts
- [ ] My own modified scripts
### Tasks
- [x] An officially su…
-
### System Info
- CPU: x86_64
- CPU mem: 64GB
- GPU name: V100 SXM2 16GB and Tesla T4 15GB both happens
- Libraries
- TensorRT-LLM commit https://github.com/NVIDIA/TensorRT-LLM/tree/3c46c2794e7f6df48…
-
I attempted to set up using a local LLM in WSL using Ollama. After entering the configuration details for the local model and restarting the app, I encountered the following error. Now, the app won’t …
-
### Checked other resources
- [X] I searched the Codefuse documentation with the integrated search.
- [X] I used the GitHub search to find a similar question and didn't find it.
- [X] I am sure that …