-
Hi!
I heard about a very promising model some while ago that you might be interested in. It's called fish.audio.
Here's a youtube demo : https://www.youtube.com/watch?v=Ghc8cJdQyKQ
Here's the…
-
**Area of Concern**
- [x] Server
- [x] Behavior of one or more Modules: Coral
- [ ] Installer
- [ ] Runtime [e.g. Python3.7, .NET]
- [ ] Module packages [e.g. PyTorch)
- [ ] Something else
**…
-
**Description**
Triton receives SIGSEGV during handling the traffic. Last thing that it wrote out was `E0723 11:57:36.328641 1 infer_handler.h:187] ""[INTERNAL] Attempting to access current response …
-
**Description**
I want to use VLMs with pytriton and vllm backend. Currently I am using sample script given at
https://github.com/triton-inference-server/pytriton/blob/main/examples/vllm/server.py
…
-
### System Info
Apple M2, Sonoma 14.6 (23G80), Python 3.12.5, pandasai 2.2.14
### 🐛 Describe the bug
The getting started example (https://docs.pandas-ai.com/library#smartdataframe) produces a wrong…
-
I have Finetuned Llama2 model with LORA for QA task and now for inference/ streaming I would like to use Triton-llm which requires TensorRT model format.
Is there any source code/ resources that I ca…
-
I run the triton server using the following commands
S3_REPO="s3://.../models/repository/"
docker run --rm --net=host --gpus=all nvcr.io/nvidia/tritonserver:23.11-py3 tritonserver --model-reposito…
-
### What feature would you like to request?
I would like to deploy fastembed as an external service, similar to [infinity](https://github.com/michaelfeil/infinity). Can we do that?
### Is there any …
-
We are currently using a _pulling update_ mechanism to get the health check.
https://github.com/containers/podman-desktop-extension-ai-lab/blob/529bc5bef181032081fb5a616c0de7afabd27c4e/packages/bac…
-
### System Info
OS: Windows 11
Rust version: cargo 1.75.0 (1d8b05cdd 2023-11-20)
Hardware: CPU AMD 6800HS
(text-generation-launcher --env didn't work)
### Information
- [ ] Docker
- [X] The CL…