search

yfzhang114 / MME-RealWorld

✨✨ MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?
79 stars 5 forks source link

Are MME-RealWorld-Base64 and MME-RealWorld consistent? #4

Open wangli68 opened 2 months ago

wangli68 commented 2 months ago

When I downloaded and used MME-RealWorld-Base64, I found that there was no "Reasoning/Remote Sensing" item in MME-RealWorld-Base64. I am not sure if only MME-RealWorld-Base64, has no "Reasoning/Remote Sensing" item or if MME-RealWorld also has a missing "Reasoning/Remote Sensing" item. MME-RealWorld-Base64 downloads from (https://huggingface.co/datasets/yifanzhang114/MME-RealWorld-Base64), MME-RealWorld downloads from (https://huggingface.co/datasets/yifanzhang114/MME-RealWorld).

yfzhang114 commented 2 months ago

Thanks for reaching out. You can check the leaderboard for details, but just to clarify, the reasoning tasks do not have a Remote Sensing split in either MME-RealWorld or MME-RealWorld-CN.