search

allenai / scifact

Data and models for the SciFact verification task.
Other
225 stars 25 forks source link

Question about Hit one #14

Closed wjddd closed 3 years ago

wjddd commented 3 years ago

Hi, I have some doubts about the metrics in verisci/evalaute/abstract_retrieval.py. Does Hit one mean that the prediction contains at least one true document id?

Thank you very much.

dwadden commented 3 years ago

Hi, we didn't end up using "hit one" in our final evaluations so I forget the details. But from what I recall, "hit one" means that the top-ranked retrieval is a correct prediction.