Closed urialon closed 1 year ago
Implementing majority@k, from the Self-Consistency paper.
I added the flags --majority_at K and --temperature X to the gsm_eval.py script.
--majority_at K
--temperature X
gsm_eval.py
Note that this leads to more predicted tokens, and thus uses the tokens-per-minute rate limits quicker.
Implementing majority@k, from the Self-Consistency paper.
I added the flags
--majority_at Kand--temperature Xto thegsm_eval.pyscript.Note that this leads to more predicted tokens, and thus uses the tokens-per-minute rate limits quicker.