search

localminimum / R-net

A Tensorflow Implementation of R-net: Machine reading comprehension with self matching networks
MIT License
323 stars 122 forks source link

anything new?? #7

Closed dengyuning closed 7 years ago

dengyuning commented 7 years ago

It seems that the result is not good. Any new progress??

ghost commented 7 years ago

Thanks for raising this issue! I've been quite busy with other works recently but I'd love to get some feedback from everyone as I might have easily misinterpreted the paper and could have written a wrong architecture. I've tried different approaches and architectures (e.g. weight sharing, 100% unidirectional RNN etc) but currently the model seems to cap at: Exact match = 40~50 F1 = 60 with mean training loss of around 2. training loss If you've reviewed the code and think it should be different please put forward a pull request! Thanks

ghost commented 7 years ago

Just clarifying confusion, the training curve above is when applied dropout of 0.2 Below is when no dropout is used. The dev performance with dropout is 0.3/0.4 for EM/F1 and without dropout is 0.4/0.5.

image

ghost commented 7 years ago

I raised another issue #12 related to this. We can keep all discussions regarding performance in this issue.