Does decoder takes encoder's last hidden and cell state as its initial state?

sunfanyunn / DARNN

A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction

112 stars 23 forks source link

Does decoder takes encoder's last hidden and cell state as its initial state? #2

Open KimMeen opened 4 years ago

KimMeen commented 4 years ago

Hi,

Thank you for sharing this implementation! I noticed that seq2seq.attention_decoder() takes encoder_state as its initial state, but the paper and one of the other implementation (Zhenye-Na/DA-RNN) initialized the decoder with zeros, so what the difference between them and why you implemented in this way?

Thank you again for the implementation.

Regards, Ming

pqy000 commented 4 years ago

Perhaps, you can try both, and see which intialization in same implementation is better?