search

rhasspy / piper

A fast, local neural text to speech system
https://rhasspy.github.io/piper-samples/
MIT License
6.24k stars 457 forks source link

SSML Support? #275

Open duplaja opened 11 months ago

duplaja commented 11 months ago

I apologize if I missed this somewhere in the documentation, but does the python install support SSML? I'm converting some of my old scripts over from Mimic3, and didn't see any flag like Mimic3's --ssml.

Alternatively, what's the best way to add a break between paragraphs, if SSML is not supported?

Thank you!

nptrainor commented 9 months ago

This would be incredibly useful.

Any thoughts on when/whether this is in train?

synesthesiam commented 9 months ago

This is planned, but I haven't made any progress yet. Adding pauses and changing the playback speed is easy, but switching voices will require more changes.

DaveXanatos commented 5 months ago

@synesthesiam Another vote for SSML here. I'm especially interested in emphasizing some words/phrases. FWIW I wrote a script that allows embedding of a voice name in the speech text string that will switch the voice. I am happy to share that if there's interest. My plan is to have a "Speech Center" up and running on ZeroMQ (or any messaging thing like MQTT, Rabbit, etc) and different scripts would be able to send text to be spoken, with embedded voice commands, to the speech center via messaging. So far - these voices are AWESOME and the script I'm using makes them incredibly simple to switch between. Thank you for making all these available.

fantnhu commented 5 months ago

That would be great :) I've been looking forward to SSML and other supplements for a long time. I don't want to use any other TTS because I am satisfied. The SSMl, pause hold and custom tags would improve the experience a lot: [laughter], [laughs], [sighs]... like in Bark TTS. If you can first solve the pause with similar parameters, that would be great, e.g. [wait=2s] Thanks for your work!

synesthesiam commented 5 months ago

I'm finally making some progress on SSML. The next version of Piper should support breaks (pauses), word/phoneme substitutions, and some say-as forms (number, date, etc.).

I can't do laughter and sighs, unfortunately. Those would have had to be present in the original datasets.

DaveXanatos commented 5 months ago

This is excellent news! I've been creating some form of emphasis by adding a slight bit of time to the --length_scale and --sentence_silence parameters, but pauses and say-as are very welcome additions!!! Thanks!

andrewfr commented 4 months ago

@synesthesiam I'm using Piper and think it is great! I too, am interested in SSM. I admit I know next to nothing about speech synthesis. I am having problems finding references on implementing SSML. Would it help to start by training a voice? Or would it be more feasible to apply a SSML feature to the audio stream? Closely connected, is how to implement "speech marks" - when a word starts and finishes in the audio.

Thanks, Andrew

nitinthewiz commented 4 months ago

Hey @synesthesiam! Looking forward to SSML support, specially to solve #401 I was trying to get lessac high to say "COVID-19" properly, but to no avail. I reckon only SSML or IPA will make it say such things correctly.

andrewfr commented 4 months ago

Hi @nitinthewiz @synesthesiam I am a newbie to all this. I have looked at #401. This may turn out to be a long shot. I am starting to learn Praat. Praat is used by linguists to study speech. Using Praat, if I can modify Piper's output to match what a SSML tag would do, that could be a start.

nptrainor commented 3 months ago

@synesthesiam - That progress sounds fabulous - thank you so much. It would be wonderful to change voices too. I am enjoying writing my own novels (stories to you and me) and using Piper to read them back to me. It is such a great help for editing etc, and having different voices for different characters would be superb.

andrewfr commented 2 months ago

@synesthesiam @nitinthewiz I am not sure if this helps. I started to play with pyDub. I needed to insert some silence to act as a break. I played with pyDub to alter an audio stream's pitch and volume. It is really dawning on me that a lot of SSML, provided one works on a sentence level, can be done in some post synthesis stage.

DaveXanatos commented 1 month ago

Much can be done post synthesis, yes, unless the synthesis needs to output real time in a responsive or conversational system (such as a robot running some type of LLM to engage with people on its environment.) This is why I'm really looking forward to SSML support in Piper.

From: andrewfr @.> Sent: Tuesday, August 20, 2024 11:47:33 PM To: rhasspy/piper @.> Cc: xanatos xanatos.com @.>; Comment @.> Subject: Re: [rhasspy/piper] SSML Support? (Issue #275)

@synesthesiamhttps://github.com/synesthesiam @nitinthewizhttps://github.com/nitinthewiz I am not sure if this helps. I started to play with pyDub. I needed to insert some silence to act as a break. I played with pyDub to alter an audio stream's pitch and volume. It is really dawning on me that a lot of SSML, provided one works on a sentence level, can done some post synthesis stage.

— Reply to this email directly, view it on GitHubhttps://github.com/rhasspy/piper/issues/275#issuecomment-2300637267, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGLWB6CEFCJ2FKTTH255AA3ZSQEVLAVCNFSM6AAAAAA7RMC5CGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBQGYZTOMRWG4. You are receiving this because you commented.Message ID: @.***>