search

MMMU-Benchmark / MMMU

This repo contains evaluation code for the paper "MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI"
https://mmmu-benchmark.github.io/
Apache License 2.0
353 stars 24 forks source link

validation_Materials_25 answer seems wrong? #27

Closed Zarjagen closed 5 months ago

Zarjagen commented 5 months ago

validation_Materials_25

Question:

The density and associated percent crystallinity for two polypropylene materials are as follows: <image 1> Compute the densities of totally crystalline.

Choices:

['0.917 g/cm^3', '0.841 g/cm^3', '0.946 g/cm^3', '0.841 g/cm^3']

Correct answer: B

image

First of all, answer B and D seem exactly the same, both are 0.841 g/cm^3. Meanwhile, according to the density formula, the final answer for totally crystalline should be around 0.943 (my own calculation), so C seems to be the correct one. Just want to report this.

xiangyue9607 commented 5 months ago

Thanks for reporting this! We will fix the error!

NipElement commented 5 months ago

We've corrected the error – the right content should be: Compute the densities of totally amorphous polypropylene, and your calculation for totally crystalline is close to the correct answer, the corresponding question is test_Materials_242. Thank you for flagging this!