rodrigonogueira4
commented
1 year ago Hi Andrew, we haven't tried query expansion in the InPars paper but my experience is that it does not make a big difference in terms of nDCG when using a reranker. However, if you are interested in a more recall-oriented task (measured by mAP), query expansion such as RM3 + reranking might help.
Table 2 from this paper shows this: nDCG@20 didn't improve with RM3 but mAP did. http://aclanthology.lst.uni-saarland.de/2020.findings-emnlp.63.pdf
When I read the paper I didn't see anything about modifying the BM25 query that pulls the initial 1000 documents before running the LLM on it. I was curious if you recommend any standard modification to it (adding synonyms, etc) before pulling the initial documents.