-
### System Info
[libprotobuf ERROR /tmp/tritonbuild/tritonserver/build/_deps/repo-third-party-build/grpc-repo/src/grpc/third_party/protobuf/src/google/protobuf/text_format.cc:335] Error parsing text-…
-
Hello, thanks for the work being done here.
**Description**
I'm trying to debug multiples issues that happens on production, and upgrading our Triton Server to 24.05 is one of the solutions i'm …
-
According to the instructions, uncommenting should enable dynamic batching, but despite uncommenting, it has not taken effect.
![企业微信截图_17029850515536](https://github.com/triton-inference-server/te…
-
**Description**
Encountered a critical issue with Triton Inference Server in poll mode where the server becomes unresponsive when loading a Python model with errors. Specifically, if a Python model h…
teith updated
6 months ago
-
## Description
The model is not reloaded when the underlying backend runtime, pytorch_backend and libtorch in this case, causes some errors.
In such cases, it would be useful in a production en…
-
### System Info
- Hardware: 8x NVIDIA H100 80GB HBM3
- Software: NVIDIA-SMI 535.129.03 Driver Version: 535.129.03 CUDA Version: 12.4
- tensorrtllm_backend commit: [d173386f4dd7b3ed5…
-
**Is your feature request related to a problem? Please describe.**
Triton has a fallback mechanism for writing intermediates to pinned CPU memory when the CUDA memory pool is full.
https://github.c…
-
**Is your feature request related to a problem? Please describe.**
I don't see any possibility to install [python_backend](https://github.com/triton-inference-server/python_backend#python-backend…
-
I installed tensorrtllm_backend in the follow way:
1. `docker pull nvcr.io/nvidia/tritonserver:23.12-trtllm-python-py3`
2. `docker run -v /data2/share/:/data/ -v /mnt/sdb/benchmark/xiangrui:/root…
-
**Description**
After compiling the triton server using installed libraries in vcpkg , these compiled libraries will cause symbol conflicts and result in the linking failure of the client compilation…