allenai / OLMo

Modeling, training, eval, and inference code for OLMo
https://allenai.org/olmo
Apache License 2.0
4.37k stars 431 forks source link

Swap in Flan with correct separators. #674

Closed dwadden closed 1 month ago

dwadden commented 1 month ago

The old Flan mix has newlines separating example inputs and outputs, which creates issues during evaluation. This version fixes it. More info here.