about encoding

[helloword12345678]

thanks for sharing. in the MnemonicReader i want to know why question and context use the same rnn encoding, why not use separate encoding,because sometime question and context may have different features.

[seanliu96]

Hi, in fact I think it is up to you. Using two RNNs is Okey due to the slight difference between the question and the context. But it is also make sense to use only one RNN. It can reduce the number of parameters to speed up. On the other hand, it is more reasonable to compute the similarity, align iteratively and fuse together.

[helloword12345678]

"compute the similarity, align iteratively and fuse together " is operated after the encoded vectors,i don't think these operators related to the RNN encoding process, can you give some more Detailed explanation ?thanks

[seanliu96]

For example,

Q: Which is the best programming language? A: ............is the best programming language......

If utilize two different RNNs, the vector of "programming language" may be very different. When computing the similarity and doing other operations, it is hard to find the locations of "programming language" in the question and answer. But if we use only one RNN, the similarity must be very high. It is easy to find the locations.

[helloword12345678]

thanks！ but except word embedding,if the question and context have different num of other features, these features and word embedding as encoder input. because of different num of features,in this situation,must use two rnn?----do you think so

[helloword12345678]

i means that: the input of the rnn encoder is three dimension: batch_sizeseq_lendim, and the dim contains both word-embedding and other external features; but sometimes question and context have different numbers of external features, in this situation,i only have to user 2 rnn encoder!------do you have any other suggestions? thanks

[seanliu96]

In fact, R-Net uses two RNNs because of the different numbers of features in questions and contexts. I think using one or two RNNs is a trade-off. If two RNNs can significantly improve the performance, why not?

[helloword12345678]

in my experiment,two rnn is worser than one rnn,even with external features

[seanliu96]

I have no idea...The architecture is hard to explain. And results can vary from data to data.

[helloword12345678]

trouble you again,you talked fuse layer. in the mnemonic reader paper is fuse layer means the aggregation layer

[seanliu96]

Oh yes. Thanks.

[helloword12345678]

trouble you again,i am puzzled to the encoder layer！ the input of it is 3-dimention (batch_sizeseq_leninput_dim) and the output is 3-dimention too。 after hearing you detailed example explanation: """ If utilize two different RNNs, the vector of "programming language" may be very different. When computing the similarity and doing other operations, it is hard to find the locations of "programming language" in the question and answer. But if we use only one RNN, the similarity must be very high. It is easy to find the locations. """ i feel that encoder layer purpose is to make similar word have similar vector,-------but if just this purpose,why it exists? just use it's input 3-dimention word embeding already Satisfy this purpose！ looking forward you explanation!!!

[seanliu96]

This is the basic idea of RNN. I recommend you to read more papers, slides or blogs.