beir-cellar / beir

A Heterogeneous Benchmark for Information Retrieval. Easy to use, evaluate your models across 15+ diverse IR datasets.
http://beir.ai
Apache License 2.0
1.54k stars 182 forks source link

Merge all latest development changes to master #98

Closed thakur-nandan closed 2 years ago

thakur-nandan commented 2 years ago

Changes added to the latest BEIR master branch:

  1. Added support for the T5 reranking model: monoT5 rerankers.
  2. Hugging Face Data loader for BEIR dataset. Uploaded all datasets on HF.
  3. MultiGPU evaluation using SBERT parallel code.
  4. Added HNSWSQ method in faiss retrieval methods.
  5. Added dependency of datasets library within setup.py
  6. Shorten README.md significantly and created a separate BEIR wiki.