search

thunlp / FewRel

A Large-Scale Few-Shot Relation Extraction Dataset
https://thunlp.github.io/fewrel.html
MIT License
729 stars 165 forks source link

How are the validation datasets used in FewRel2.0 #60

Closed lanzhuzhu closed 3 years ago

lanzhuzhu commented 3 years ago

Hi, I found there are some more validation datasets than what is mentioned in your paper for FewRel2.0, such as val_nyt.json, val_pubmed.json. I am not sure how these datasets are used?
These datasets are not used in your experiments for paper FewRel2.0, are they? Are there any papers that used these datasets?

Thanks!

gaotianyu1350 commented 3 years ago

Hi,

val_pubmed.json is used for FewRel 2.0. You can see this as the validation set of the domain adaptation challenge. However, val_nyt.json is not used in the FewRel 2.0 paper. This dataset is annotated on NYT corpus, and people can use it however they want.

lanzhuzhu commented 3 years ago

Hi,

val_pubmed.json is used for FewRel 2.0. You can see this as the validation set of the domain adaptation challenge. However, val_nyt.json is not used in the FewRel 2.0 paper. This dataset is annotated on the NYT corpus, and people can use it however they want.

Thanks for your prompt reply. I guess val_pubmed.json should be used as the validation set of the domain adaptation challenge. What confused me is the description in the paper "FewRel2.0:***". It says that val_semeval.json is the validation set for that challenge. Thus , I guess it would be better to clarify this somewhere for this challenge.

Thanks again!