search

NVIDIA / TensorRT

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
https://developer.nvidia.com/tensorrt
Apache License 2.0
10.72k stars 2.12k forks source link

Does TensorRT support accelerating Tacotron model inference? #169

Closed superhg2012 closed 4 years ago

superhg2012 commented 5 years ago

There are some Bi-LSTM modules in Tacotron model(TTS network) and some convs that are wrapped, does tensorrt support this?

rmccorm4 commented 4 years ago

Hi @superhg2012,

  1. For tacotron2, there's a Pytorch implementation: https://github.com/NVIDIA/tacotron2 which you might be able to export to ONNX, and try to convert to TensorRT, but I haven't tried it myself.

  2. For tacotron implemented in TensorFlow, like so: https://github.com/keithito/tacotron

I would think you can use TF-TRT, where TRT-compatible nodes will be sped up and incompatible nodes will fallback to the TF implementation. I did find one post on a user trying this but failing here: https://devtalk.nvidia.com/default/topic/1062601/tensorrt/fail-to-speed-up-model-by-tensorrt-/ but that might be different now.

  1. You could also try the same as (2) above for tacotron2 implemented in Tensorflow, maybe like this one: https://github.com/Rayhane-mamah/Tacotron-2