Is majority voting(self-consistency) helpful for 70B llama2-sft model?

OFA-Sys / gsm8k-ScRel

Codes and Data for Scaling Relationship on Learning Mathematical Reasoning with Large Language Models

215 stars 16 forks source link

Is majority voting(self-consistency) helpful for 70B llama2-sft model? #15

Open platoonpluto opened 1 year ago

GanjinZero commented 1 year ago

We have not enough resources to complete that experiment. From my experience, using a slightly larger temperature (maybe larger than 0.7) will be helpful with self-consistency for a large SFT model.