TTS usage? - Githubissues

ggerganov / whisper.cpp

Port of OpenAI's Whisper model in C/C++

MIT License

34.84k stars 3.55k forks source link

TTS usage? #2146

Closed abdelazizSalah closed 4 months ago

abdelazizSalah commented 4 months ago

I have implemented this well as Speech to text model, however, I heard that whisper can also work as text to speech model, can I use this repo and these models as text to speech also, or this is only for speech to text?

magnacartatron commented 4 months ago

Hi @abdelazizSalah No it can't. What OpenAI refers as to whisper in their API docs when they mention TTS isn't the same as the whisper model that is being used here. Some time in 2022 OpenAI was kind enough to provide their models on HuggingFace. The community to this date is still using said models or refined versions. These are not the same as the Whisper TTS that OpenAI uses for its conversations. Coqui AI TTS was a solid TTS but they went out of business. You can still use their models (see Huggingface). Otherwise there is Suno Bark but your milage may vary there. TTS models are so far fairly closed which is frustrating to say the least. MacOS allows you to install various high quality Siri voices, and then you can just use "say 'I want a pizza'" in the terminal and generate audio.

abdelazizSalah commented 4 months ago

Okay, thanks for this!
I have tried to use AVSpeechSynthesis, using the following code provided in Apple documentation
however, it did not work, and it always gives me an error saying : "cannot load assets folder"
And when I searched more, I found that there are some issues saying that there is a leakage in this model, and it does not work n the modern ios versions, so do you know any work around to make it work, or how to find a TTS model to use in swiftUI ?

magnacartatron commented 4 months ago

@abdelazizSalah why dont you just use say. If you're on a Mac just install a Siri voice you like via setting and set it as the default. Then you can run "say 'hello there'" in the terminal and you'll get OS to play a fairly good TTS output. If you're using node you can spawn a process to run that command through your application etc. you can use -o to output to a file. If you don't want to mess around it's the fastest easiest high quality tts I can find.

abdelazizSalah commented 4 months ago

@magnacartatron Yes it worked!, thanks a lot