-
**Description**
I'm trying to deploy text to speech model with onnx and triton. When running the server, I get this error: failed:Protobuf parsing failed.
also model status is : UNAVAILABLE: Interna…
-
Seeing failures in sm-python-sdk tests, related to either no credentials, or no Docker daemon.
* - `botocore.exceptions.NoCredentialsError: Unable to locate credentials`
* - FileNotFoundError: [Er…
-
Hi,
I am interested in implementing MLflow in my project, where I have built several speech models and NLP-based machine translation (MT) models. I am looking to incorporate continuous training and…
-
As indicated by the title, on the main branch, I used 40 threads to simultaneously send inference requests to the in-flight Triton Server, resulting in the Triton Server getting stuck.
The specifi…
-
### **Problem:**
When using model-analyzer with --triton-launch-mode=remoted, I encounter connectivity issues.
### **Context:**
I have successfully started Triton Inference Server on the same ser…
-
### System Info
Docker image: nvcr.io/nvidia/tritonserver:24.07-trtllm-python-py3
Device: 8x H100
trt-llm backend: v0.11.0
### Who can help?
@byshiue @schetlur-nv
### Information
- [ ] The off…
-
Hello,
When trying to run the tritonserver on a setup with 4 nodes, I face an failure that seems to suggest a mismatch between the number of GPUs per node and the tensor parallel (TP) * pipeline para…
-
### System Info
4*NVIDIA L20
### Who can help?
_No response_
### Information
- [X] The official example scripts
- [ ] My own modified scripts
### Tasks
- [ ] An officially suppor…
-
python web_demo_mm.py -c "/data/shared/Qwen/models" --share --server-name 0.0.0.0 --server-port 80
/usr/local/lib/python3.8/dist-packages/auto_gptq/nn_modules/triton_utils/kernels.py:411: FutureWarn…
-
**Description**
triton infer server with the docker image not working on Jetson Orin NX 16 GB JP 5.1.1
See attached log
[server.log](https://github.com/triton-inference-server/server/files/152630…