🐸 TTS roadmap - Githubissues

erogol commented 3 years ago

These are the main dev plans for :frog: TTS.

If you want to contribute to :frog: TTS and don't know where to start you can pick one here and start with our Contribution Guideline. We're also always here to help.

Feel free to pick one or suggest a new one.

Contributions are always welcome :muscle: .

v0.1.0 Milestones

[x] Better model config handling #21
[x] TTS recipes for public datasets.
[x] TTS trainer API to unify all the model training scripts.
[x] TTS, Vocoder and SpeakerEncoder model abstractions and APIs.
[x] Documentation for
- [x] Implementing a new model using :frog: TTS.
- [x] Training a model on a new dataset from gecko.
- [x] Using Synthesizer interface on CLI or Server.
- [x] Extracting Spectrograms for Vocoder training.
- [x] Contributing a new pre-trained :frog: TTS model.
- [x] Explanation for Model config parameters/

v0.2.0 Milestones

[x] Grapheme 2 Phoneme in-house conversion. (Thx to gruut 👍 )
[x] Implement VITS model.

v0.3.0 Milestones

[x] Implement generic ForwardTTS API.
[x] Implement Fast Speech model.
[x] Implement Fast Pitch model.

v0.4.0 Milestones

[x] Trainer API v2 - join the discussion
[x] Multi-speaker VCTK recipes for all the TTS.tts models.

v0.5.0 Milestones

[x] Support for multi-lingual models
[x] YourTTS release 🚀

v0.6.0 Milestones

[x] Add ESpeak support
[x] New Tokenizer and Phonemizer APIs #937
[x] New Model API #1078
[x] Splitting the trainer as a separate repo 👟Trainer
[x] Update VITS model API
[x] Gradient accumulation. #560 (in 👟)

v0.7.0 Milestones

[x] Implement Capacitron 👑 @a-froghyar 👑 @WeberJulian
[x] Release pretrained Capacitron

v0.8.0 Milestones

[x] Separate numpy transforms
[x] Better data sampling for VITS
[x] New Thorsten DE models 👑 @thorstenMueller

🏃‍♀️ Milestones along the way

[ ] Implement End-to-end training API for ForwardTTS models a vocoder. #1510
[ ] Implement a Python voice synthesis API.
[ ] Inject phonemes to the input text at inference. #1452
[ ] AdaSpeech1/2 https://arxiv.org/pdf/2104.09715 and https://arxiv.org/abs/2103.00993
[ ] Let the user pass a custom text cleaner function.
[ ] Refactor the text cleaners for a more flexible and transparent API.
[ ] Implement HifiGAN2 (not the vocoder)
[ ] Implement emotion and style adaptation.
[ ] Implement FastSpeech2 (https://arxiv.org/abs/2006.04558).
[ ] AutoTTS 🤖 (👑 @loganhart420)
[ ] Watermarking TTS outputs to sign against DeepFakes.
[ ] Implement SSML v0.0.1
[ ] ONNX and TorchScript model exports.
[ ] TensorFlow run-time for training models.

🤖 New TTS models

[x] AlignTTS (@erogol)
[x] HiFiGAN (#16 :crown: @rishikksh20 and @erogol)
[x] UnivNet Vocoder ( :crown: @rishikksh20)
[x] VITS paper
[x] FastPitch source
[x] Alignment Network paper
[x] End2End TTS combining aligner + tts + vocoder.
[x] Multi-Lingual TTS (#11 :crown: @WeberJulian )
[ ] ParallelTacotron paper (open for contribution)
[ ] Efficient TTS paper (open for contribution)
[ ] Gaussian length regulator from https://arxiv.org/pdf/2010.04301.pdf (open for contribution)
[ ] LightSpeech from https://arxiv.org/pdf/2102.04040.pdf (open for contribution)
[ ] AdaSpeech1/2 https://arxiv.org/pdf/2104.09715 and https://arxiv.org/abs/2103.00993

erogol commented 1 year ago

We are not working on SSML currently, it is back in the list without a precise timeline.

offside609 commented 1 year ago

Please do!!

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. You might also look our discussion channels.

violet17 commented 10 months ago

Will you support bark-small? Thanks.

csukuangfj commented 8 months ago

Any plan to a port of coqui-ai engine for android? TTS on android is very robotic (espeak, rhvoice, festival lite).

@paolo-caroni

Please take a look at https://github.com/coqui-ai/TTS/discussions/3194

You can use sherpa-onnx to run VITS models from Coqui on Android and also embedded devices, e.g., raspberry pi.

We have pre-built Android APKs for the VITS English models from Coqui. https://k2-fsa.github.io/sherpa/onnx/tts/apk.html

DmitryVN commented 8 months ago

Fix it plz https://github.com/coqui-ai/TTS/issues/3039 https://github.com/coqui-ai/TTS/issues/3282 The problem persists and because of this, normal correct use is not possible. Also at the moment it kind of breaks off the phrase at the end of each sentence and it turns out a jerky reading.

MarkChrisE2091 commented 7 months ago

Any new update?

csukuangfj commented 7 months ago

Any plan to a port of coqui-ai engine for android? TTS on android is very robotic (espeak, rhvoice, festival lite).

@paolo-caroni

We have supported it in https://github.com/k2-fsa/sherpa-onnx/pull/508

The following is a YouTube video https://www.youtube.com/watch?v=33QYuVzDORA

You can use all coqui-ai/TTS models and piper models listed in https://github.com/k2-fsa/sherpa-onnx/releases/tag/tts-models with https://github.com/k2-fsa/sherpa-onnx/pull/508

imevro commented 3 months ago

hi guys, why?

upd: found https://twitter.com/_josh_meyer_/status/1742522906041635166

NicoleKai commented 3 months ago

Their ability to exist and be profitable was dependent on how much better their tech was compared to everyone else. It may not feel like it, but we are in the middle of an AI singularity. Coqui's business model might have stood a chance if they started with this tech 5 years earlier, but it was probably too little too late. Eleven labs is probably eating their lunch :/

angadkwatra73 commented 1 month ago

Can we still use their product well, even though they have shut down?

eginhard commented 1 month ago