How do I evaluates the miracl dataset with the Ndcg@10 metric?

FlagOpen / FlagEmbedding

Retrieval and Retrieval-augmented LLMs

MIT License

6.8k stars 487 forks source link

How do I evaluates the miracl dataset with the Ndcg@10 metric? #714

Open daegonYu opened 4 months ago

daegonYu commented 4 months ago

Is there any code that evaluates the miracl dataset with the Ndcg@10 metric? Or, I know that in order to evaluate Ndcg on the miracl dataset, the similarity between positives and negatives needs to be labeled. How do I know how to do labeling?

staoxiao commented 4 months ago

You can refer to: https://github.com/castorini/pyserini?tab=readme-ov-file . And the evaluation of bge-m3 will be integrated into this repo in the future.