-
### System Info
colab T4
### Who can help?
@
### Information
- [X] The official example scripts
- [ ] My own modified scripts
### Tasks
- [X] An officially supported task in the `examples` fold…
-
are we allowed to use a local LLM (on-prem)?
-
Hi TensorRT-LLM team, Your work is incredible.
By following the READme file for [multi-modeling](https://github.com/NVIDIA/TensorRT-LLM/blob/main/examples/multimodal/README.md), we were sucess to run…
-
### Version
VisualStudio Code extension
### Operating System
Windows 11
### What happened?
When using Mac, Linux or Windows 11 with WSL2 Ubuntu, I get the following Bug.
Whenever a longer outp…
-
Please add ability to use Locally hosted LLM, for example using LM Studio
Getting far away from Closed Source LLM is the best for everyone.
-
### Question Validation
- [X] I have searched both the documentation and discord for an answer.
### Question
So I tried running a model like this llm = VllmServer(
api_url="http://localhost:80…
-
{"type": "error", "data": "Community Report Extraction Error", "stack": "Traceback (most recent call last):\n File \"/home/zippo/GraphRAG/ollama/repo/graphrag-local-ollama/graphrag/index/graph/extrac…
-
### Describe the bug
get the following error
`Loading llama3.1...
Traceback (most recent call last):
File "/opt/homebrew/bin/interpreter", line 8, in
sys.exit(main())
^…
-
I've been setting up the ChatQnA AIPC from the opea-project on my system and have followed the instructions provided here: https://github.com/opea-project/GenAIExamples/tree/main/ChatQnA/docker/aipc
…
-
I am interested in using Uptrain with a locally hosted open-source LLM as evaluator LLM. I'm currently hosting a LLM service using vLLM, not with Ollama. Is there any way to use this local LLM with U…