search

beir-cellar / beir

A Heterogeneous Benchmark for Information Retrieval. Easy to use, evaluate your models across 15+ diverse IR datasets.
http://beir.ai
Apache License 2.0
1.54k stars 182 forks source link

Support for revisions in HFDataLoader #116

Open Muennighoff opened 1 year ago

Muennighoff commented 1 year ago

Would be great if the HFDataLoader would support dataset revisions.

Also imo it would be much better to merge the qrels HF repos with the regular ones & load the entire dataset via .load_dataset() using a dataset script on the repos.