search

NVIDIA-AI-IOT / whisper_trt

A project that optimizes Whisper for low latency inference using NVIDIA TensorRT
Other
63 stars 9 forks source link

whisper_trt with Triton on a Jetson board #8

Open brantapl opened 3 months ago

brantapl commented 3 months ago

Awesome project. Do you know if a whisper_trt model can be served in Triton on a Jetson Orin board?