Inference starting repeat itself.

NVIDIA / flowtron

Flowtron is an auto-regressive flow-based generative network for text to speech synthesis with control over speech variation and style transfer

https://nv-adlr.github.io/Flowtron

Apache License 2.0

887 stars 177 forks source link

Inference starting repeat itself. #139

Closed Bahm9919 closed 2 years ago

Bahm9919 commented 2 years ago

First of all, thanks guys for your great work and support. I have some problem with flowtron. Especially, the model repeating itself. And provide only 4sec sentence. Even if sentence has 1 sec. So its repeating itself all 4 sec. Warmstart using Tacotron2(130k) which doesnt has this problem. And i got good results. Starting for 1 flow step. c1e93e8c-6dba-4bd1-8e80-887a2f1681e9 Screenshot_1 Screenshot_2

Hope for your helping Rafael. @rafaelvalle

Bahm9919 commented 2 years ago

Solved. Need to train gate from scratch like in README.

nikich340 commented 2 years ago

Solved. Need to train gate from scratch like in README.

How much epochs did you do? Also, what dataset did you use?

Bahm9919 commented 2 years ago

Solved. Need to train gate from scratch like in README.

How much epochs did you do? Also, what dataset did you use?

i'm using my own collecting russian dataset. For this, it was about 50 epochs. Need train not only gate, so gate doesnt matter for this problem, need train without attention, and training nflow2 solved this problem, and training with cummulative attention.

nikich340 commented 2 years ago

Solved. Need to train gate from scratch like in README.

How much epochs did you do? Also, what dataset did you use?

i'm using my own collecting russian dataset. For this, it was about 50 epochs. Need train not only gate, so gate doesnt matter for this problem, need train without attention, and training nflow2 solved this problem, and training with cummulative attention.

So you did full retraining from scratch? No way to fix it for pretrained models?

Bahm9919 commented 2 years ago

Solved. Need to train gate from scratch like in README.

How much epochs did you do? Also, what dataset did you use?

i'm using my own collecting russian dataset. For this, it was about 50 epochs. Need train not only gate, so gate doesnt matter for this problem, need train without attention, and training nflow2 solved this problem, and training with cummulative attention.

So you did full retraining from scratch? No way to fix it for pretrained models?

There is several ways to fix it.

use your checkpoint path for gate training like in README. Training from scratch. Step 4.
use checkpoint path for warmstart n_flow2,
use checkpoint path of n_flow2 for warm start with cummulative_attention. (very recommended) Also you can set use_attn_prior to false and continue training from your checkpoint path.