search

triton-inference-server / tensorrtllm_backend

The Triton TensorRT-LLM Backend
Apache License 2.0
664 stars 96 forks source link

only python grpc client can cancle request, when will support with golang grpc #433

Closed jiuweisu closed 5 months ago

jiuweisu commented 5 months ago

the doc descript only python support request cancel.we use golang grpc.

https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/user_guide/request_cancellation.html?highlight=cancel#

byshiue commented 5 months ago

Please ask this question in tritonserver repo.