search

thunlp / NRE

Neural Relation Extraction, including CNN, PCNN, CNN+ATT, PCNN+ATT
MIT License
809 stars 309 forks source link

The pr.txt files does not match the curve in the paper #5

Closed pfllo closed 8 years ago

pfllo commented 8 years ago

Hi,

I'm trying to reproduce the PR curves in the paper. However I find that the pr.txt files in the repository does not match the curves reported in the paper (PCNN+ATT for example).

Are these files generated by models that are not fully trained? Can you provide pr.txt files that can reproduce curves in the paper?

Much appreciated.

SharmisthaJat commented 8 years ago

I have the same question. pr-1.txt has 2000 lines, shouldn't it have more than ~96k lines (more than one prediction for each entity pair in test file). I changed line 237 of test.h to {for (int i=0; i<int(aa.size()); i++)} which results in ~96k*52 lines in pr.txt. But, the graph still does not match. I am plotting column 2 and 3 (recall and precision I assume) of pr-1.txt for the curve, is that the correct interpretation?

It will be very helpful if you could provide us with a script to generate the PR curve.

Thanks

Mrlyk423 commented 8 years ago

I have updated the test.h in PCNN+ATT. And the original pr.txt and models of our experiments can be downloaded in https://pan.baidu.com/s/1bo0fgXx.

pfllo commented 8 years ago

Thank you! @Mrlyk423 This helps a lot!

SharmisthaJat commented 8 years ago

Thanks for your response