search

bigcode-project / bigcode-evaluation-harness

A framework for the evaluation of autoregressive code generation language models.
Apache License 2.0
698 stars 180 forks source link

ignore --use_auth_token if model doesn't require it #227

Open Vipitis opened 2 months ago

Vipitis commented 2 months ago

Came across this today, and seems like it would be a little improvement.

if --use_auth_token is set but no token is available (not logged in to huggingface-cli) the program will aborts with 

huggingface_hub.utils._headers.LocalTokenNotFoundError: Token is required (`token=True`), but no token found.

Even if the model specified is not gated. So should this arg be ignored and the model loaded anyway? Essentially making it okay to always include this arg. However this somehow makes this arg redundant.

can gladly open a PR to try and implement this

or is this actual behaviour that could be addressed in transformers itself?