search

openai / grade-school-math

1.05k stars 148 forks source link

The third example problem is incorrect #4

Open melaniebeck opened 1 year ago

melaniebeck commented 1 year ago

image

"With one person drinking 5, that brings the total drank to 5+9+8+3=25" is incorrect according to the problem setup and should instead be 5+9+8=22.

The accompanying paper estimates that fewer than 2% of problems have errors like the one above. Unless this example was explicitly cherry-picked to demonstrate an incorrect problem, it is either exceedingly unlucky that this problem made it into the showcased examples or the frequency of incorrect examples is likely higher.

zsc commented 1 year ago

Thanks for pointing out! Fortunately, they got it correct in the train.jsonl anyway.