search

k2-fsa / sherpa

Speech-to-text server framework with next-gen Kaldi
https://k2-fsa.github.io/sherpa
Apache License 2.0
483 stars 97 forks source link

[Triton] Update recipe to build TRT engine and model_repo for Conformer offline model #432

Closed wd929 closed 1 year ago

wd929 commented 1 year ago

Hi all,

This PR updates recipes to build TRT engine and a unified model_repo for Conformer offline model. I added additional TRT building script using same model_repo structure as ONNX model with several parameters changing. FP16 ONNX model can be used when set fp16=true, where we use polygraphy convert to perform the FP32->FP16 I added one audio feature for TRT inference testing. Also updated the README for Triton.

Thanks!

csukuangfj commented 1 year ago

@yuekaizhang

Thanks for reviewing. Please merge it if you think it is ready.

yuekaizhang commented 1 year ago

Many thanks! Merge it.