-
**Description**
In a ensemble pipeline for TensorRT-LLM backend, when we try to propagate data from preprocessing model to the postprocessing model, we get this error **Model 'ensemble' receives inpu…
-
First of all, thank you very much for your outstanding work. In my task, I successfully replaced the Mamba1 module and encountered the following problem during the backward of the Mamba2 module. How c…
xypjq updated
1 month ago
-
### System Info
L4 GPU
GPU memory: 24 GB
TensorRT LLM version: v0.10.0
container used: tritonserver:24.06-trtllm-python-py3
### Who can help?
@byshiue @schetlur-nv
### Information
- [X] The …
-
/usr/bin/ld: ../libtritonserver.so: undefined reference to `absl::lts_20220623::StartsWithIgnoreCase(absl::lts_20220623::string_view, absl::lts_20220623::string_view)'
/usr/bin/ld: ../libtritonserv…
-
**Description**
I encounter a crash when I am using big model with ONNX backend on CPU. The problem seems to be related to this closed ticket: https://github.com/triton-inference-server/server/issu…
-
Experiencing this error when starting a sagemaker endpoint with local-cache:
`error: creating server: Invalid argument - unable to find 'libtritoncache_local.so' for cache. Searched: /opt/tritonserve…
-
**Description**
bug when deploying Macbert
**Triton Information**
I use the official image: nvcr.io/nvidia/tritonserver:21.09-py3
```
NVIDIA Release 21.09 (build 27443074)
Copyright (c) 20…
-
When can NAV support creating Triton Repo for this new backend? Is it on your roadmap?
https://github.com/triton-inference-server/tensorrtllm_backend
-
Hello.
I am writing to inquire about the PyTorch version used in the Triton Inference Server 24.01 release.
Upon reviewing the documentation, I noticed that Triton 24.01 includes PyTorch version…
-
**Description**
Thanks for this remarkable work, i deploy model with a variable execpt input tesnor. So i wanna to send this variable via query_params during each infer request.
But i can not find…