StonyBrookNLP / ircot

Repository for Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions, ACL23
https://arxiv.org/abs/2212.10509
Apache License 2.0
173 stars 21 forks source link

metric/ SupportEmF1Metric(Metric) #31

Open sjk0825 opened 1 month ago

sjk0825 commented 1 month ago

hi, I have recall question.

In your source code SupportEmF1Metric(Metric), your program get title-level and para-level. about this criterion, I have some question

  1. para mean sentence in context? each passage in Teutberga title? ex. ['Teutberga', ['Teutberga( died 11 November 875) was a queen of Lotharingia by marriage to Lothair II.', "She was a daughter of Bosonid Boso the Elder and sister of Hucbert, the lay- abbot of St. Maurice's Abbey."]]

  2. the recall score in paper is title-level? or passage level?

thank you