Rayhane-mamah
commented
6 years ago Hello again @butterl, thanks for your interest in this project.
I am in the same spot as you for now and being the impatient person I am, I didn't settle for a slower convergence speed that the actual.. Accoding to this comment, output should become audible in about 5k steps. (Natural synthesis with no teacher forcing!). Actually as soon as the alignments become right (3k steps), the audio become audible. All steps after that are to correct pronunciation details (e.g: "I" and "Y" in the previously linked comment).
Since I don't have any GPU available currently, I didn't push the training further than 8k steps. Audio quality is slightly better than the primary samples but there are still some pronunciation issues.
So, I can't really say for sure how many steps are needed, since at this point the model enters the "slow learning" part, my titan XP should however be here by the end of the week, I'll tell you how it goes (and probably put some pretrained Ljspeech and M-AILABS en_US models).
You can perfectly stop the training whenever you like (ctrl+c) and come back to it later. For evaluation, there is the logs-Tacotron folder which contains training logs if you are concerned about audio quality. There are also the mel and alignment plots which can help you judge the state of the model. As if to do some synthesis, maybe on totally different sentences from those in the training data, you can run the synthesize.py with --mode='eval' and it will evaluate on the test sentences inside hparams.py.
Note:
- If the audio quality is what concerns you the most, I am currently working on the wavenet vocoder which should give a human like voice at the price of generation speed.
- If however, generation speed is much more important than the human like quality as long as quality is acceptable, I can add a post processing network to the tacotron to predict linear spectrograms directly. This gives a nice overall audio quality and a fast generation. (keitito's few samples here)
- Further approaches will be adopted later to speed up generation (e.g: WaveRNN) but that will be for a later time.
If there is anything else, I can assist you with, please let me know!
butterl
imdatceleste
I'm using the default training dataset（LJSpeech-1.1） and got no GPU on my machine .The CPU training speed is about 7.182 sec/step， for fast envaluation ，how many steps do you think it would be OK to see different（big enough performance different） between the other TTS？
And since the training is running， is it OK is pause（ctrl+c） when I want to stop and envalute ?