KoljaB / RealtimeTTS

Converts text to speech in realtime
1.39k stars 119 forks source link

Voice clone/ change voice #70

Closed Yokopops closed 2 months ago

Yokopops commented 2 months ago

Hello, starting to learn a little bit using your pyqt6_speed_test.py, changed it a little bit so it only processes after pressing a submit button and also clear the text box after processing. Are you able to assist me with changing/cloning the Coqui voice while still using this script?

Kindest regards.

KoljaB commented 2 months ago

Of course.

Have a look at the FAQ and whatever is unclear just ask

Yokopops commented 2 months ago

I apologies for overlooking the FAQ, I feel bad for asking now, got it working though! Want to try and work out how to add a dropdown to select a voice like how you can select one of the engines, unsure how to go about it though as I'm still learning, if you have any sources of information on how to do that I would be very grateful.

KoljaB commented 2 months ago

Sure, this is not covered in the docs. Every engine has a get_voices() method which returns objects of a voice class depending to the engine. AzureEngine returns AzureVoice, ElevenlabsEngine ElevenlabsVoice etc. Every of these Voices have a name Parameter. You can use this name to submit as parameter to set_voice method which also every engine has. Or you submit the object of the Voice class.

KoljaB commented 2 months ago

Usage of get_voices: https://github.com/KoljaB/RealtimeTTS/blob/master/example_fast_api/server.py#L422-L428

Usage of set_voice: https://github.com/KoljaB/RealtimeTTS/blob/master/example_fast_api/server.py#L290-L304

Yokopops commented 2 months ago

Will have a go, thanks for the info. Really trying my hardest on this to help other people that are like me.

Yokopops commented 2 months ago

Struggling to get it to work, maybe I'm misunderstanding the command?

    # Set voice based on dropdown selection
    selected_voice = self.voice_dropdown.currentText()
    self.engine_coqui.set_voice(selected_voice)

This is the code I am attempting to use from the info I pulled from your examples. Sorry if I'm overlooking something, I'm really trying to learn, appreciate your patience.

Yokopops commented 2 months ago

After some more tinkering I managed to load a voice from my main script! never realized I have to use it like this: self.engine_coqui = CoquiEngine(voice=selected_voice), time for sleep now.

KoljaB commented 2 months ago
    # Set voice based on dropdown selection
    selected_voice = self.voice_dropdown.currentText()
    self.engine_coqui.set_voice(selected_voice)

Looks good. If self.engine_coqui contains an object of CoquiEngine that should work.

self.engine_coqui = CoquiEngine(voice=selected_voice)

This would create another engine instance every time, if you call it on a change of the dropdown item. This takes a lot of time and requires the old engine instance to be properly closed with the shutdown method. I would not implement it that way. Creating one CoquiEngine instance before and then calling set_voice method on that like in the first example would be the "correct" approach.

Yokopops commented 2 months ago

Having another go now, it would appear it is not changing the voice when using

self.engine_coqui = CoquiEngine()
# Set voice based on dropdown selection
selected_voice = self.voice_dropdown.currentText()
self.engine_coqui.set_voice(selected_voice)

however when I changed self.engine_coqui = CoquiEngine() to self.engine_coqui = CoquiEngine(voice=selected_voice) it does load the selected voice, however not from the dropdown, obviously. I have even attempted to make a simple button to send self.engine_coqui.set_voice(voice="voice1.wav")

Not really sure where I'm going wrong though, sorry for being a pain..

Yokopops commented 2 months ago

OMG IM SO DUMB! I was calling the voices with the .wav added on got it working now, sorry.

KoljaB commented 2 months ago

Not your fault, I wasn't aware it doesn't work with ".wav" added. It shouldn't be this way and also it isn't documented. Thanks for pointing this out, I'll have a look at this. It should definitely also work with ".wav" at the end.

Yokopops commented 2 months ago

Oh, don't feel as silly now, Is it possible to change the model somehow to use a different one?

Love playing around with this code to make something actually useful to me though. Keep up the good work, its awesome!