Why use the 'tf.layer.conv1d' for query, key transformation instead of fully connected layer?

syang1993 / gst-tacotron

A tensorflow implementation of the "Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis"

368 stars 110 forks source link

Why use the 'tf.layer.conv1d' for query, key transformation instead of fully connected layer? #36

Open LEEYOONHYUNG opened 5 years ago

LEEYOONHYUNG commented 5 years ago

Considering that the query and key have a timestep of 1 from the perspective of 1d convolution, I think that it is totally same with using the fully connected layer.

Is there any reason for using the tf.layer.conv1d for query, key transformation instead of fully connected layer?