NoMIRACL: Knowing When You Don't Know for Robust Multilingual Retrieval-Augmented Generation

[anoopkunchukuttan]

Paper: https://arxiv.org/abs/2312.11361 Repo: https://github.com/project-miracl/nomiracl

Languages: bn, hi, te

• A testset to evaluate whether a paragraph contains an answer to a query
• Use for evaluating hallucinations and error-rates in RAG systems
• However, the dataset does not contain responses for queries - just whether the retrieved passages are relevant or not
• Good for retrieval evaluation
• Monolingual set i.e. query and passage in same language - good to evaluating multilingual retrieval and IR

[anoopkunchukuttan]

Miracl paper