How to merge document lists retrieved from multiple queries？

jzbjyb / FLARE

Forward-Looking Active REtrieval-augmented generation (FLARE)

MIT License

545 stars 50 forks source link

How to merge document lists retrieved from multiple queries？ #6

Closed sunxiaojie99 closed 11 months ago

sunxiaojie99 commented 11 months ago

Hi～ In the paper, it is mentioned that "We retrieve using each generated question and interleave the returned documents into a single ranking list to aid future generations." I am curious about how to integrate document lists under multiple queries into a unique ranking list.

jzbjyb commented 11 months ago

Thanks for the question! We interleave multiple ranking lists into a single list. If there're two queries q1 and q2, each with three returned documents:

q1 list: d1, d2, d3
q2 list: d4, d1, d5

the merged list has 5 documents because d1 is duplicate:

d1, d4, d2, (d1 is skipped since it is duplicate) d3, d5

It's implemented in the openai_api.py file.