openai / grade-school-math

992 stars 145 forks source link

Ambiguity in 217th train data #11

Open ruoxining opened 1 year ago

ruoxining commented 1 year ago

Question: Ravi has some coins. He has 2 more quarters than nickels and 4 more dimes than quarters. If he has 6 nickels, how much money does he have?

Gold ans: 350

As the question does not specify the unit (cent), it will be judged as false if the model predicts the answer to be 3.5 dollars, which actually equals the correct answer of 350 cents. The question will be better if it is expressed as "how many cents does he have?" thus the model knows in which unit to output the answer. We face such a problem in developing our model and hope it to be fixed. Thank you!