Problem of the Softmax on Read Mode

ixaxaar / pytorch-dnc

Differentiable Neural Computers, Sparse Access Memory and Sparse Differentiable Neural Computers, for Pytorch

MIT License

335 stars 56 forks source link

Closed yat011 closed 5 years ago

yat011 commented 5 years ago

Should the softmax be applied on the last dimension? (i.e. the dimension of the read mode)

Currently, each read mode would always return 1 if the model has only one read head.

ixaxaar commented 5 years ago

That sounds logical, though lemme check the paper and get back on this.

ixaxaar commented 5 years ago

Hi @yat011 can you perhaps review the changes?

yat011 commented 5 years ago

Hi @ixaxaar, It looks good! Thanks a lot.