The Additive-Attention has always been bad.
According your paper, this Attention helps Forward Attention to align, so is it normal that always bad?
My Forward Attention is well aligned.
I use all of the VCTK, and the batch_size is 32.
The figure below shows the number of epochs 37, 47, and 48.
I got the answer in "Investigation Of Enhanced Tacotron Text-To-Speech Synthesis Systems With Self-Attention For Pitch Accent Language" this paper, additive-attention is work.
Thanks.
The Additive-Attention has always been bad. According your paper, this Attention helps Forward Attention to align, so is it normal that always bad? My Forward Attention is well aligned.
I use all of the VCTK, and the batch_size is 32. The figure below shows the number of epochs 37, 47, and 48.