search

keras-team / keras

Deep Learning for humans
http://keras.io/
Apache License 2.0
62.09k stars 19.48k forks source link

Memory network - 100% on babi task 1 #1418

Closed farizrahman4u closed 7 years ago

farizrahman4u commented 8 years ago

This is a pre - PR issue. Recently, I managed to come up with a model which gives 100% on the babi task single supporting fact 10k in 30 epochs. But each epoch takes 40 seconds on CPU. Current example takes only 3 seconds per epoch, but requires 120 epochs to reach 97.5%. Also, the model deviates too much from those described in the paper. Should I PR it? If yes, should I update the existing example or add a new one?

fchollet commented 8 years ago

Facebook also reported 100% accuracy on that task with MemNN. So I assume the 97.5% MemNN example could be optimized further to get to 100%.

If you have an architecture that is sufficiently different, I recommend you test it on all the babi tasks, and compare your results to what Facebook reports for MemNN. You could also try to apply it to a different QA dataset, like DeepMind's.

You could submit it as a separate example, but for the sake of diversity it would be preferable that it be on a different dataset or at least a different babi task. Including the DeepMind QA dataset in Keras would be neat I think.

farizrahman4u commented 8 years ago

Ok. I will do a MemNN example on deepmind's dataset. The accuracy of babi_rnn example is too low. So in the mean time I will try to reduce my Memory network to a basic rnn without loss(for task 1, you can unroll memory network to a simpler model and still get same results) and PR babi_rnn.py.