search

Rayhane-mamah / Tacotron-2

DeepMind's Tacotron-2 Tensorflow implementation
MIT License
2.28k stars 905 forks source link

Loss exploded when training WaveNet #460

Open wjqkkky opened 4 years ago

wjqkkky commented 4 years ago

I use GTA synthesis by Tacotron-2 to predict mel spectrograms and then train the wavenet vocoder with default hparams. I have two problems: 1.The loss < 0 2.When iterating after 196k the training exited due to exception: Loss exploded.

Has anyone ever been in this situation?And what should i do?Thanks a lot!

1583420625(1) 1583420691(1)

poor1017 commented 4 years ago

I have encounter this problem: exploded. Have you solved it? Thanks!

gdineshk6174 commented 4 years ago

@wjqkkky hi, can you share wavenet eval plot of waveform of both original and predicted from last saved checkpoint.