How the soft attention model be implemented in this project?

kracwarlock / action-recognition-visual-attention

Action recognition using soft attention based deep recurrent neural networks

http://www.cs.toronto.edu/~shikhar/projects/action-recognition-attention

350 stars 157 forks source link

How the soft attention model be implemented in this project? #7

Closed kyuusaku closed 8 years ago

kyuusaku commented 8 years ago

Hi, @kracwarlock. I am so confused about the implementation of the soft attention model. Why the codes related to alpha (pstate & pctx) in the lstm_cond_layer function are different from the equation (4) in your paper? Hope you can give more explanations about how the weights mappingW_i to the ith element of the location softmax be implemented in this project. Thanks a lot.

kracwarlock commented 8 years ago

The alpha variable was exactly supposed to be the location softmax mentioned in equation (4) in the paper and is calculated here: https://github.com/kracwarlock/action-recognition-visual-attention/blob/master/src/actrec.py#L274-L277

We ended up using information from both h_{t-1} and x_t so the paper equation should now have (W_i h_{t-1} + Wc_i x_t) instead of (W_i h_{t-1}). I missed making this change in the paper and will correct the paper whenever we push a new revision.