network state Ambiguity

QingyaoAi / Deep-Listwise-Context-Model-for-Ranking-Refinement

A Tensorflow implementation of the Deep Listwise Context Model (DLCM) for ranking refinement.

Apache License 2.0

134 stars 56 forks source link

network state Ambiguity #2

Closed hh23333 closed 5 years ago

hh23333 commented 6 years ago

Hi QingyaoAi, Thanks for releasing code, it seems that there is an ambiguity in regard to the meaning of St(new state) in your paper and code. In the paper, St in formula(4) means new_gate in GRU. But in your code it seems that St is the last timestep hidden outputs: encoder_outputs, encoder_state = tf.nn.static_rnn(enc_cell, encoder_embed, dtype=dtype) line 664 of RankLSTM_model.py

Can you explain it to me? Thanks!

QingyaoAi commented 6 years ago

Thanks for your question!

The encoder_state is the the network state vector outputted by tf.nn.static_rnn, which is then used in rnn_decoder as the initial_state. The hidden outputs of rnn is stored in encoder_outputs.

hh23333 commented 6 years ago

Thanks for you reply！

The out put of tf.nn.static_rnn: Returns: A pair (outputs, state) where:

outputs is a length T list of outputs (one for each input), or a nested tuple of such elements.
state is the final state where output(encoder_outputs in your code) is T timesteps GRU last layer hidden outputs, and state(encoder_state in your code) is the last timesteps GRU hidden outputs vector of all layers.

In your paper the network state st and sn is the newgate of GRU, not the last timesteps GRU hidden outputs. So my question is, is there a difference between the presentation of paper and the implementation of the code.

QingyaoAi commented 6 years ago

I see. I didn't look at the details of GRU in Tensorflow when I implemented our model. If Tensorflow implements the GRU as the way you described it, then the notation in the paper is a little bit misleading. Thanks for pointing this out.