search

Mostafa-Samir / DNC-tensorflow

A TensorFlow implementation of DeepMind's Differential Neural Computers (DNC)
MIT License
581 stars 164 forks source link

Sequence length in dnc/dnc.py #2

Open jhyuklee opened 7 years ago

jhyuklee commented 7 years ago

While implementing bAbI task based on your code, I've found that the sequence length is defined as a single integer for each batch in dnc.py.

For copy task, you fed the same length items for each batch (though its random), but if each item in one batch varies in their length (like sentences in bAbI), shouldn't it be array of sequence lengths? (size of batch)

Mostafa-Samir commented 7 years ago

Yes, it should be an array of sequence lengths, that's indeed a current limitation in the implementation, and it's mentioned explicitly in the usage guide and implicitly in the to-dos.

The reason behind going with such a limiting design is performance as I'm using a kid of modest hardware. but I'm planning to remove that limitation, so I'll keep this issue open till then. However, this shouldn't be an obstacle for the bAbI task, as per the paper, their best performing model used a batch of size 1.

jhyuklee commented 7 years ago

Oh I've missed that their batch size was 1. Thank you for your kind reply!