Question about evaluation

boudinfl / ake-datasets

Large, curated set of benchmark datasets for evaluating automatic keyphrase extraction algorithms.

Apache License 2.0

142 stars 29 forks source link

Question about evaluation #1

Closed tagucci closed 5 years ago

tagucci commented 5 years ago

Hi, thanks for awesome repo! I am a new to keyphrase extraction and would like to know whether I should evaluate my own model on test.author.json, test.readers.json or test.combined.json on NUS/SemEval2010/etc. On the other hand, there are test.contr.json and test.uncontr.json on Inspec. In your MultipartiteRank paper, you used NUS and SemEval2010. Which json file did you used to evaluate your model in the paper?

thanks.

boudinfl commented 5 years ago

Hi @tagucci,

In the MultipartiteRank paper, and as in most previous works, the test.combined.json (i.e. authors+readers) and the test.uncontr.json (uncontrolled vocabulary) are used for SemEval-2010/NUS and Inspec evaluation.

Best regards,