search

awslabs / rag-qa-arena

Apache License 2.0
28 stars 2 forks source link

Different docu id #5

Open Yeeebye opened 1 month ago

Yeeebye commented 1 month ago

The document id in annotation.._with citation.json seems to be different with the document id in ..from_colbert_test.json

Any reason to that, or how should I find the documents that are used in the answers?

Yeeebye commented 1 month ago

For example, the gold_docu_id for the first question in lifestyle is 102494, but it's not in lifestyle_from_colbert_test.jsonl given by you, nor in the dev_from_colbert_test.jsonl I got from running the script.

rujunhan commented 2 weeks ago

Hi, sorry for the late response. Please just use the one in the annotation.._with_citation.json as they are created directly from the underlying corpus.

You can also verify by comparing the text directly. @KaiserWhoLearns to comment more here. Thanks.