bigcode-project / bigcode-evaluation-harness

A framework for the evaluation of autoregressive code generation language models.
Apache License 2.0
744 stars 193 forks source link

add trust_remote_code to tokenizer loading #71

Closed loubnabnl closed 1 year ago

loubnabnl commented 1 year ago

some tokenizers require running local code hence the need for trust_remote_code argument