Fixed memory candidate: multiplication => addition

jimfleming / recurrent-entity-networks

TensorFlow implementation of "Tracking the World State with Recurrent Entity Networks".

https://arxiv.org/abs/1612.03969

MIT License

273 stars 65 forks source link

Fixed memory candidate: multiplication => addition #17

Closed liufly closed 7 years ago

liufly commented 7 years ago

Thank you for providing the TensorFlow implementation. One minor fix.

As per Equation (3) in the original paper, it should be addition: h_j^~ <- \phi(U h_j + V w_j + W s_t) rather than multiplication: h_j^~ <- \phi(U h_j * V w_j * W s_t)

This can be further verified in their Torch code: local candidate_memories = nn.PReLU(opt.edim)(nn.CAddTable(){U, V, W}):annotate{name = 'prelu'}

jimfleming commented 7 years ago

Weird. I broke that in one of my updates (c601af04a4a9dbcab5e3a2af93a4019b1e96a13e). Thanks for the PR!