arkilpatel / SVAMP

NAACL 2021: Are NLP Models really able to Solve Simple Math Word Problems?
MIT License
116 stars 34 forks source link

Some bad data with wrong answers in this dataset #13

Open jwmueller opened 5 months ago

jwmueller commented 5 months ago

Some of the examples in this dataset have the wrong answer.

I automatically detected some of these with my company's software, and shared them here: https://huggingface.co/datasets/Cleanlab/bad_data_gsm8k_svamp.csv

Example:

Question: Rachel’s tree had 4 apples. She picked 2 apples from her tree. Thereafter 3 new apples grew on the tree. How many apples are there on the tree now?

Answer According to the Dataset: 1