lupantech / MathVista

MathVista: data, code, and evaluation for Mathematical Reasoning in Visual Contexts
https://mathvista.github.io/
Creative Commons Attribution Share Alike 4.0 International
197 stars 28 forks source link

Wrong answer to problems #12

Closed WesleyHsieh0806 closed 4 months ago

WesleyHsieh0806 commented 4 months ago

The answers to several problems seem incorrect, I guess some info are missing for the question?

For example, the below is Problem 950:

question 950: The slope of f(x) at x=0 is ____
answer: positive
choices: ['positive', 'negative', 'zero', 'undefined']

image

First, the function in the image is g(x) not f(x). Second, even if g(x) = f(x), the slope should be negative. The relation between g(x) and f(x) is probably missing, e.g., g'(x) = f(x)?

lupantech commented 4 months ago

Thanks for your feedback!

Despite our efforts in reviewing the benchmark multiple times, it's inevitable that a small number of examples may contain incorrect or ambiguous information. Considering the need for consistent evaluations, we've decided to retain these examples in the current version. However, we are planning to release a new version to correct the errors identified.