search

openai / grade-school-math

1.05k stars 148 forks source link

Some bad data with wrong answers in this dataset #25

Open jwmueller opened 6 months ago

jwmueller commented 6 months ago

Some of the examples in this GSM8K dataset have the wrong answer.

I automatically detected some of these with my company's software, and shared them here: https://huggingface.co/datasets/Cleanlab/bad_data_gsm8k_svamp.csv

Example:

Question: After scoring 14 points, Erin now has three times more points than Sara, who scored 8. How many points did Erin have before?

Answer According to the Dataset: 18

johnowhitaker commented 1 month ago

Can confirm all 5 examples flagged seem wrong