search

substratusai / kubeai

AI Inference Operator for Kubernetes
https://www.kubeai.org
Apache License 2.0
533 stars 43 forks source link

Consider adding TensorRT engine #192

Open nstogner opened 2 months ago

nstogner commented 2 months ago

Opening issue to collect information on whether there is a good reason to add TensorRT as a serving backend.

https://github.com/NVIDIA/TensorRT-LLM/issues/334

viningz commented 2 months ago

+1