NLU augmentation for Query generation and Doc expansion

[manisnesan]

Improved text ranking with few shot prompting

• This blog post discusses using large language models (LLMs) to generate labeled data for training ranking models.
• The authors propose using a handful of human-annotated labeled examples and prompting the LLM to generate synthetic queries for documents in the corpus.
• This approach can create high-quality synthetic training data at a low cost, which can be used to train much smaller and compute-efficient ranking models.
• The authors describe the process of generating labeled data via instruction-prompting LLMs and explain how to improve the model's generated output using prompt engineering.
• They also provide experimental results showing the effectiveness of this approach on a biomedical dataset

[manisnesan]

UDAPDR - Unsupervised Domain Adaptation via LLM Prompting and Distillation for Rerankers

Related Demonstrate Search Predict architecture , Promptagator++

• https://github.com/primeqa/primeqa/tree/main/extensions/udapdr/scripts

[manisnesan]

An extension of the blogpost Jo Kristian Bergum - Boosting Ranking performance using minimal supervision - YouTube

[manisnesan]

Sujitpal also showcased this idea in haystack US 2023 talk

https://github.com/manisnesan/til/issues/41#issuecomment-1575046998

[manisnesan]

Query Expansion with Cohere

[manisnesan]

Query2Doc

[manisnesan]

Smaller Open Source LLMs for Query Expansion

smaller variants of T5 and UL2 outperform BM25 on datasets like BEIR.

Even in zero-shot settings.

UL2's Recall@1K on BEIR: • 72.43 for BM25 • 74.47 for Query2Doc • 72.86 for Query2Expansion • 75.07 for Chain of Thought

Source

[manisnesan]

Measure Query Rewrite Quality