search

chaitanyamalaviya / ExpertQA

[Data + code] ExpertQA : Expert-Curated Questions and Attributed Answers
MIT License
118 stars 13 forks source link

Missing evidence passages #5

Open yumoxu opened 8 months ago

yumoxu commented 8 months ago

Hello authors, thank you for the great work!

Quick question regarding the released data data/r2_compiled_anon.jsonl: it seems many entries have URLs only for claim["evidence"], rather than URL+ passage.

I was wondering if this means the evidence is inaccessible and the claim thus falls into the support=="N/A" category. But this does not seem to be always true after further inspection.

Evidence passages are necessary for model evaluation (please correct me if I am wrong). Could you share the best way to retrieve them for these URLs? Thank you!

chaitanyamalaviya commented 8 months ago

Hi @yumoxu , thanks for your interest! We added the file with all evidence passages at data/r2_compiled_all_evidences_autoais_anon.jsonl. Let us know if you have any more questions.