tonytan48 / KD-DocRE

Implementation of Document-level Relation Extraction with Knowledge Distillation and Adaptive Focal Loss
110 stars 19 forks source link

frequent/longtail relation #23

Open JZBZ2020 opened 1 year ago

JZBZ2020 commented 1 year ago

hi, can you tell me how to differentiate frequent and long-tail relations in section 3.5 of this paper? Moreover, infer-f1?

tonytan48 commented 1 year ago

Hi, the frequent F1 refers to the top 10 out of 96 relations in DocRED (accounts for roughly 60%), and long-tail refers to the rest of the relations. For infer-F1, this is evaluating on a subset of multi-hop relation triples. We did not introduce infer-F1 and we just followed the protocol from https://aclanthology.org/2020.emnlp-main.127/.

wucui5 commented 1 year ago

Hello, I would like to replicate your experiments regarding the long-tail relationships and frequent relationships. Could you please provide the relevant datasets? I have already processed the dev.json and train_annotated.json files myself. However, I'm facing difficulties with the test.json file. The results I obtained using the original test.json file and the processed long-tail or frequent relationship files are significantly different from yours. I would greatly appreciate any assistance you can provide.

YjwHello commented 1 year ago

Hi, the frequent F1 refers to the top 10 out of 96 relations in DocRED (accounts for roughly 60%), and long-tail refers to the rest of the relations. For infer-F1, this is evaluating on a subset of multi-hop relation triples. We did not introduce infer-F1 and we just followed the protocol from https://aclanthology.org/2020.emnlp-main.127/.

请问这个infer-F1是怎么计算的,多跳关系三元组的子集是怎么生成的