HFDataLoader is incompatible with CQADupstack

beir-cellar / beir

A Heterogeneous Benchmark for Information Retrieval. Easy to use, evaluate your models across 15+ diverse IR datasets.

http://beir.ai

Apache License 2.0

1.54k stars 182 forks source link

HFDataLoader is incompatible with CQADupstack #114

Open Muennighoff opened 1 year ago

Muennighoff commented 1 year ago

If I see correctly cqadupstack is the only dataset not uploaded to the Hub, i.e. there's only its qrels. This prevents using it with DPRES :( Would be great if it could be uploaded!