ixaxaar / pytorch-dnc

Differentiable Neural Computers, Sparse Access Memory and Sparse Differentiable Neural Computers, for Pytorch
MIT License
335 stars 56 forks source link

Question about allocation weighting #59

Open PeterDeWachter1998 opened 2 years ago

PeterDeWachter1998 commented 2 years ago

The paper describes the allocation weighting vector as:

$a_t[\phi_t[j]] = (1 - u_t[\phi_t[j]]) \prod u_t[\psi_t[j]] $

In your part of the code where you calculate the right-part product you do this:

v = var(sorted_usage.data.new(batch_size, 1).fill_(1))
cat_sorted_usage = T.cat((v, sorted_usage), 1)
prod_sorted_usage = T.cumprod(cat_sorted_usage, 1)[:, :-1]

Why do you create the var "v", which contains "ones", and concatenate it? This does not seem the same as in the paper.

Thanks, Peter