search

p0p4k / vits2_pytorch

unofficial vits2-TTS implementation in pytorch
https://arxiv.org/abs/2307.16430
MIT License
465 stars 81 forks source link

Added ONNX Export and Inference #43

Closed w11wo closed 10 months ago

w11wo commented 10 months ago

Tested ONNX Export and Inference with LJSpeech-no-sdp. Commands are as follows:

python export_onnx.py --model-path="G_64000.pth" --config-path="config.json" --output="vits2.onnx"
python infer_onnx.py --model="vits2.onnx" --config-path="config.json" --output-wav-path="output.wav" --text="hello world, how are you?"
p0p4k commented 10 months ago

Good PR, very helpful.

w11wo commented 10 months ago

Solved merge conflict @p0p4k

p0p4k commented 10 months ago

Any metric for CPU vs GPU vs ONNX infer time? Thanks a lot!

DavidSamuell commented 10 months ago

I've conducted tests and measured the different inference speeds between CPU, GPU, ONNX CPU, and ONNX GPU. The results are as follows:

Model: Pytorch CPU: 2.3481 seconds Model: Pytorch GPU: 0.1600 seconds Model: ONNX CPU: 2.1909 seconds Model: ONNX GPU: 0.1367 seconds

The ONNX model is just slightly faster than its PyTorch counterpart. You can find the Colab notebook here.

@p0p4k