Add audio streaming support

DanRuta / xVA-Synth

Machine learning based speech synthesis Electron app, with voices from specific characters from video games

GNU General Public License v3.0

587 stars 54 forks source link

Add audio streaming support #67

Open Pendrokar opened 5 months ago

Pendrokar commented 5 months ago

@DanRuta ~~had mentioned that v0.12 of TorchAudio has streaming capabilities. If TorchAudio was xVASynth's was updated, perhaps it would easily allow implementing audio streaming?~~ https://pytorch.org/blog/pytorch-1.12-new-library-releases/#beta-streaming-api

[Edit] Misinterpreted, while xVASynth does install TorchAudio, it is only used for mel spectrogram representation

lexkoro commented 4 months ago

What would be the use case of this?

Pendrokar commented 4 months ago

What would be the use case of this?

This would allow real-time generation of audio for Text-To-Speech and Speech-To-Speech (STS). STS also known as Voice Conversion. Playback of incomplete files depends on the use audio player or game audio engine. So while xVASynth is currently most used by Skyrim's Mantella mod, Skyrim itself is not able to playback incomplete files.