search

bigcode-project / bigcode-evaluation-harness

A framework for the evaluation of autoregressive code generation language models.
Apache License 2.0
795 stars 214 forks source link

Fix stop words #50

Closed Muennighoff closed 1 year ago

loubnabnl commented 1 year ago

Thanks for the fix! I think we could still have some cases of left stop tokens since we get the start length by batch but this should make it better for datasets with very different prompts sizes.