-
## Implementation
- [ ] Look into LiteLLM (OpenAI, Anthropic, OpenAI Azure support) + Ollama
- [ ] Add LLM connection setup to user settings + global default
- [ ] Replace OpenAI setup popup by LLM…
-
A Personal NotebookLM and Perplexity-like AI Assistant for Everyone. Research and Never forget anything.
[Github](https://github.com/MODSetter/SurfSense)
Key Features
💡 Idea: Have your own private …
-
[TensorRT-LLM] TensorRT-LLM version: 0.13.0
0.13.0
^M0it [00:00, ?it/s]^M139it [00:00, 1375.80it/s]^M201it [00:00, 1554.11it/s]
[1729020016.135793] [toyota-tom-buddy-ml-vm:879 :0] ucp_context.c:1…
-
TRT-LLM version: v0.11.0
I'm deploying a bart model with medusa heads, and i notice this issue https://github.com/NVIDIA/TensorRT-LLM/issues/1946, then i adapted my model with follow steps:
```
1…
-
Great project, and thank you for sharing!! I am wondering whether you have any insight about how to build this kind of system using local llms? Maybe using something like https://github.com/mlc-ai/web…
-
are we allowed to use a local LLM (on-prem)?
-
I have successfully built and started docker container for tensorrt_llm and ran the convert_checkpoints.py as well as trtllm_build as follows:
1. docker run -it --net host --shm-size=4g --name triton…
-
After I had installed paper-qa
```
mamba create -n paperqa2 python=3.11
mamba activate paperqa2
git clone https://github.com/Future-House/paper-qa.git
cd paper-qa
pip install -e .
```
...a…
-
### System Info
a100
### Who can help?
@byshiue
@juney-nvidia
### Information
- [ ] The official example scripts
- [x] My own modified scripts
### Tasks
- [x] An officially supported task in th…
-
How to support the new model in cpp runtime ? Is there any reference document ? For example, the multimodal model [llava-one-vision](https://huggingface.co/lmms-lab/llava-onevision-qwen2-7b-ov)
Foll…