texttron / tevatron

Tevatron - A flexible toolkit for neural retrieval research and development.
http://tevatron.ai
Apache License 2.0
494 stars 94 forks source link

Miracl dataset support #60

Closed jasper-xian closed 1 year ago

jasper-xian commented 1 year ago

Trying to use miracl/miracl datasets in tevatron generates this error:

FileNotFoundError: Dataset 'miracl/miracl' doesn't exist on the Hub. 
If the repo is private, make sure you are authenticated with `use_auth_token=True` after logging in with `huggingface-cli login`.

We can fix this by adding a requirement use_auth_token=True whenever we load a dataset from huggingface.