-
### System Info
4*NVIDIA L20
### Who can help?
_No response_
### Information
- [X] The official example scripts
- [ ] My own modified scripts
### Tasks
- [ ] An officially suppor…
-
**Description**
I was using Triton Server nvcr.io/nvidia/tritonserver:24.04-py3 on my local machine with Windows 10 via docker container. Ie installed latest Nvidia Driver 555.85, and docker containe…
-
When we inspect triton server logs we see entries like this:
```
I1011 13:21:57.174321 1 cache_manager.cc:174] Creating TritonCache with name: 'local', libpath: '/opt/tritonserver/caches/local/libt…
-
**Description**
I'm trying to deploy text to speech model with onnx and triton. When running the server, I get this error: failed:Protobuf parsing failed.
also model status is : UNAVAILABLE: Interna…
-
### System Info
When using Qwen2, executing inference with the engine through the run.py script outputs normally. However, when using Triton for inference, some characters appear garbled, and the out…
-
Hello,
I am currently experiencing an issue with the `triton-inference-server/tensorrt_backend` while trying to run a Baichuan model.
### Description
I have set `gpt_model_type=inflight_fused…
-
**Description**
A clear and concise description of what the bug is.
before calling unloadmodel，memory isbelow：
and after calling unloadmodel，memory isbelow：
**Triton Information**
What vers…
-
**Description**
Triton Server with Pytorch Backend build not working for CPU_ONLY. It is expecting libraries like libcudart.so even though the build was for CPU. Below is how we invoke the build. Fro…
-
**Description**
Triton Sever crashed after some period of time running inferences using Python Backend models. The Python backend models are running TensorRT models with [mmdeploy python api](https:/…
-
**Description**
yolov8n.pt模型转torchscripts和onnx，在triton server或Deepytorch Inference上推理，精度下降
**Triton Information**
What version of Triton are you using?
nvcr.io/nvidia/tritonserver:23.04-py3
Are…