search

beir-cellar / beir

A Heterogeneous Benchmark for Information Retrieval. Easy to use, evaluate your models across 15+ diverse IR datasets.
http://beir.ai
Apache License 2.0
1.54k stars 182 forks source link

DPR [KILT] checkpoint release plan and which data is used? #100

Open hanseokOh opened 2 years ago

hanseokOh commented 2 years ago

Hello, thanks for the great work and open-sourced code! I have questions about DPR [KILT] model which is recorded on BEIR retrieval leaderboard.

First, what specific KILT data is used for this model? Is it the same setting as Multi-task Retrieval for Knowledge-Intensive Tasks (8 knowledge-intensive IR tasks)?

image

Second, do you have a plan to release the checkpoint for this model?

Thanks!