search

bigcode-project / bigcode-evaluation-harness

A framework for the evaluation of autoregressive code generation language models.
Apache License 2.0
698 stars 180 forks source link

How can I pass num_beams? #237

Closed Sunt-ing closed 3 weeks ago

Sunt-ing commented 1 month ago

Cannot find a way to pass num_beams. Thanks for your help!

loubnabnl commented 3 weeks ago

Hi there's no flag for num_beams as we either use greedy or top-p sampling but you can change the code to add it here https://github.com/bigcode-project/bigcode-evaluation-harness/blob/f0f2b52ab0bac95b7fa881693e82c5ddcf2f9e95/bigcode_eval/generation.py#L68

Sunt-ing commented 3 weeks ago

Thanks!