triton-inference-server / tensorrtllm_backend

The Triton TensorRT-LLM Backend
Apache License 2.0
588 stars 81 forks source link

[Documentation improvement] Improve README for tensorrtllm_backend - v0.8.0 #395

Open kelkarn opened 2 months ago

kelkarn commented 2 months ago

https://github.com/triton-inference-server/tensorrtllm_backend/tree/v0.8.0

The README for this Triton server version has many references to the 23.10 version of Triton, which I believe based on the support matrix, does not support v0.8.0. v0.8.0 is only supported on 24.02 and 24.03 based on what I can see. Can we improve the documentation here please?

Screenshot 2024-04-08 at 10 48 19 AM Screenshot 2024-04-08 at 10 48 46 AM