search

joschu / cgt

Computation Graph Toolkit
Other
628 stars 87 forks source link

NTM documentation #36

Closed EderSantana closed 8 years ago

EderSantana commented 8 years ago

In the NTM example you mention that

rprev_bhm:          previous vector read from memory. [-1, 1]

but this vector is the result of a linear operation between the memory and the address vector. But the memory is not bounded, how the read vector got bounded? Shouldn't it be [-inf, inf] as well?

joschu commented 8 years ago

Hi Eder, you're right -- I'll update the docs. (Of course, maybe we should apply a tanh after reading, but that's it's empirical question whether that makes a difference.)