Hi there,
Congrats on the nice work. I am one of the authors of REBEL (https://github.com/Babelscape/rebel) and we compared our work with yours on the DocRED end-to-end Relation Extraction setting.
Due to a recent issue in our repo (https://github.com/Babelscape/rebel/issues/26), I realized that our comparison may be unfair to your work. Basically, in our work we considered the first mention to an entity in order to extract the triplets in a seq2seq fashion. I had assumed that in JEREX, a relation was correct if both entities were correct but didn't occur to me that you considered the entities correct only if all mentions are extracted.
Ie. in your example from Table 3, would it be considered correct if the last mention to Melville wouldn't have been predicted?
If not, then I believe it is unfair to directly compare REBEL with JEREX, as your evaluation is stricter, and I would try to clarify it.
Hi there, Congrats on the nice work. I am one of the authors of REBEL (https://github.com/Babelscape/rebel) and we compared our work with yours on the DocRED end-to-end Relation Extraction setting.
Due to a recent issue in our repo (https://github.com/Babelscape/rebel/issues/26), I realized that our comparison may be unfair to your work. Basically, in our work we considered the first mention to an entity in order to extract the triplets in a seq2seq fashion. I had assumed that in JEREX, a relation was correct if both entities were correct but didn't occur to me that you considered the entities correct only if all mentions are extracted.
Ie. in your example from Table 3, would it be considered correct if the last mention to Melville wouldn't have been predicted?
If not, then I believe it is unfair to directly compare REBEL with JEREX, as your evaluation is stricter, and I would try to clarify it.
Thanks, Pere-Lluis.