search

tarepan / SpeechMOS

Easy-to-Use Speech MOS predictors
MIT License
215 stars 16 forks source link

Inference acceleration #19

Closed marianasignal closed 2 months ago

marianasignal commented 3 months ago

Very good work, is there any way to speed up the reasoning? For example assembling a batch for batch inference?

tarepan commented 3 months ago

Thanks for your interest!

As you metioned, sample batching speed up inference.
The method is here.

Another method is no-gradient inference.
This docs will help you.