-
### System Info
the offical docker env in docker/Dockerfile.multi of commit "c629546"
### Who can help?
@byshiue @ncomly-nvidia
I try to convert deepseek-v2-lite
```
python convert_checkpoint.py …
-
I deployed Qwen2.5-14B-Instruct on my local server and started llm correctly using vllm.
But when I executed the sample code,
```
from paperqa import Settings, ask
local_llm_config = dict(
…
-
### System Info
- CPU: x86_64, Intel(R) Xeon(R) Platinum 8470
- CPU/Host memory size: 1TB
- GPU:
4xH100 96GB
- Libraries
TensorRT-LLM: main, 0.15.0 (commit: b7868dd1bd1186840e3755b97ea3d3a73dd…
-
I followed the exact instructions provided by TensorRT-LLM to setup triton-llm server for whisper
I am stuck with the following error when i try to build the TRT:
```
[TensorRT-LLM] TensorRT-LLM ve…
-
**Describe the bug**
HuggingFace datasets imported by nemo.collections.llm.gpt.data in /opt/NeMo/nemo/collections/llm/gpt/data/__init__.py:15
`from nemo.collections.llm.gpt.data.dolly import DollyDat…
-
What about custom/private LLMs. Will there be an option to use some of longchain local features like llama.cpp?
-
Hi, thanks for building and opening Savvy!
Is there any way I can configure it to use a locally-running LLM? With OpenAI-compatible API or otherwise.
Thanks!
-
My code：
```
import typing as t
import asyncio
from typing import List
from datasets import load_dataset, load_from_disk
from ragas.metrics import faithfulness, context_recall, context_precisi…
-
Is there any plan to support local offline models?
-
- [LMStudio](https://lmstudio.ai/)
- [Continue](https://continue.dev/docs/intro)