search

ixaxaar / pytorch-dnc

Differentiable Neural Computers, Sparse Access Memory and Sparse Differentiable Neural Computers, for Pytorch
MIT License
335 stars 56 forks source link

Question about your examples #55

Open mng8 opened 4 years ago

mng8 commented 4 years ago

Why do you not use controller hidden or read vectors over time as your perform the task?

ixaxaar commented 4 years ago

The examples are from the paper.

Personally, I did use the controller hidden vectors as well as let the memory remain uncleared for entire training time while doing various experiments myself and many other variations of these.

mng8 commented 4 years ago

Looking at inside the model code, I do see you use the reads and controller hidden vector inside the time iterations

ixaxaar commented 4 years ago

Yes, one has to use them within timesteps otherwise the whole thing is just a bunch of vanilla RNNs i think.

What I was talking about is entire training time across batches and epochs, for e.g. using the memory as a global knowledge base of sorts.