search

slczgwh / REDN

Downstream Model Design of Pre-trained Language Model for Relation Extraction Task
MIT License
95 stars 14 forks source link

The comparison with previous SOTA seems unfair #15

Closed 131250208 closed 3 years ago

131250208 commented 4 years ago

From the original paper, the NYT and WebNLG used by CasRel (HBT) separately contain 56195 + 5000 + 5000 sentences and 5019 + 500 + 703 sentences，which are inconsistent with the ones (69710, 31511) mentioned in your paper. It is unfair to compare with the results copied from the paper when using different datasets because the improvements may come from more training data. So, I recommend that reproduce HBT results on your datasets or use the datasets from HBT, and then give a more persuasive result.

slczgwh commented 3 years ago

When we published this paper, HBT was not open sourced yet, so we did not know their dataset split and used Opennre's data split and original WebNLG dataset. If your insterested in this question, you can try yourself.