search

collabora / WhisperSpeech

An Open Source text-to-speech system built by inverting Whisper.
https://collabora.github.io/WhisperSpeech/
MIT License
3.54k stars 185 forks source link

Who is Tonya Beige? (and not John Gault)... #119

Closed BBC-Esq closed 3 months ago

BBC-Esq commented 3 months ago

I'm seeing some shiny new models on huggingface, including a "fast" version of "small" and other stuff...Was just curious what the differences are and if there's going to be some explanation in the readme or somewhere? The commit notes say "Tonya Beige" and stuff like that. lol.

jpc commented 3 months ago

Hehe, the names come from faker a Python library generating random but readable names. I am afraid there are no mysterious ex-girlfriends as someone suggested on Discord ;)

The new models are trained with a reversed balance between encoder and decoder layers (so 3/4th of layers in the encoder). This makes them significantly faster when generating speech and also more robust. Unfortunately they also take 10x longer to train...

I need to start updating the README since there are new docs scattered around the place, fine-tuning scripts are also almost ready...

BBC-Esq commented 3 months ago

Hehe, the names come from faker a Python library generating random but readable names. I am afraid there are no mysterious ex-girlfriends as someone suggested on Discord ;)

The new models are trained with a reversed balance between encoder and decoder layers (so 3/4th of layers in the encoder). This makes them significantly faster when generating speech and also more robust. Unfortunately they also take 10x longer to train...

I need to start updating the README since there are new docs scattered around the place, fine-tuning scripts are also almost ready...

Cool, thanks for the clarification. That was me on the Discord BTW, or did you already know that? Anyhow, excited to see what WhisperSpeech is coming out with next. It's become a side hobby of mine to experiment and program.

In that vein, did you get a chance to look at the pull request I did? It's been over a week and that's outside your normal turnaround time for these kinds of things. ;-) No pressure!

Close this issue whenever until/unless Tonya Beige returns.