search

google-deepmind / AQuA

A algebraic word problem dataset, with multiple choice questions annotated with rationales.
Other
293 stars 43 forks source link

problem with dataset answer #1

Open joocjun opened 2 years ago

joocjun commented 2 years ago

Hello I was looking through your test dataset and found that the answer to the question on the line 153 of test.json looks wrong to me. 120=8x should give x=15 but not only is the answer shown C)10, I can't even find 15 in the given choice.

question: "If Jill needed to buy 10 bottles of soda for a party in which 8 people attended, how many bottles of soda will she need to buy for a party in which 12 people are attending?" choices:["A)6", "B)8", "C)10", "D)12", "E)14"] rationale: "We can set up a proportion to solve:\n10 bottles / 8 people = x bottles / 12 people.\nCross-multiply to solve a proportion:\n(10)(12) = (8)(x)\n120 = 8x\n10 = x\nAnswer :C." correct: C

joocjun commented 2 years ago

There also seems to be a problem involving duplicate choices in the test set. {"question": "A paper is in a square form whose one side is 20 cm. Two semi circles are drawn on its opposites as diameters. If these semi circles are cut down what is the area of the remaining paper?", "options": ["A)8.75", "B)8.79", "C)8.75", "D)8.71", "E)8.72"], "rationale": "(5 * 3.5)/2 = 8.75\nAnswer:C", "correct": "C"} A & C have the same values.