search

AndreaCossu / Relation-Network-PyTorch

Implementation of Relation Network and Recurrent Relational Network using PyTorch v1.3. Original papers: (RN) https://arxiv.org/abs/1706.01427 (RRN): https://arxiv.org/abs/1711.08028
MIT License
19 stars 7 forks source link

The accuracy is very low #4

Closed iamxpy closed 5 years ago

iamxpy commented 5 years ago

I cloned the repo and finished the preliminary work (in Prerequisites), then simply used the following command to run the code: python launch_rrn_babi.py --epochs=50000 --cuda

and here is part of the log:

......

Epoch  49000  /  50000
Train loss:  2.410700951099396 . Validation loss:  1.8214629411697387
Train accuracy:  0.16200000000000014 . Validation accuracy:  0.17250000000000024

Epoch  49500  /  50000
Train loss:  2.3995521495342254 . Validation loss:  1.8207665807008744
Train accuracy:  0.16660000000000014 . Validation accuracy:  0.17250000000000024

Epoch  50000  /  50000
Train loss:  2.405573818922043 . Validation loss:  1.8204271459579469
Train accuracy:  0.16240000000000024 . Validation accuracy:  0.16099999999999998

End training!
Testing...
Test accuracy:  0.15699999999999983
Test loss:  1.8213920128345489

What can I do to improve the accuracy? Thanks in advance for any info you can provide!

AndreaCossu commented 5 years ago

Hi @iamxpy and thank you for reaching me out. Unfortunately I only run few experiments within this project before getting absorbed by another one (currently ongoing). I have experienced the issue you reported and I want to dig further into it as soon as I have time. In the meantime feel free to experiment with it: as an example, I was going to change the embedding representation of a word with one-of-k representation as the vocabulary size of babi is very limited.
If you try some variations let me know!

AndreaCossu commented 5 years ago

I I made significant changes to the code. The Relation Network is now working as expected (need little more tuning to match paper's results). From now on, I plan to focus more on Recurrent Relational Networks. Hopefully I will be able to provide a working version soon enough.

iamxpy commented 5 years ago

I I made significant changes to the code. The Relation Network is now working as expected (need little more tuning to match paper's results). From now on, I plan to focus more on Recurrent Relational Networks. Hopefully I will be able to provide a working version soon enough.

Amazing! Thanks for your work and kindly sharing!