gpt4v refuse to answer/ insist on "I'm sorry, but I'm unable to view images" these kind of things

MMMU-Benchmark / MMMU

This repo contains evaluation code for the paper "MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI"

https://mmmu-benchmark.github.io/

Apache License 2.0

353 stars 24 forks source link

gpt4v refuse to answer/ insist on "I'm sorry, but I'm unable to view images" these kind of things #19

Closed SweetGUOguo closed 8 months ago

SweetGUOguo commented 8 months ago

Thanks a lot for your effort!!! Can I ask if you are experiencing gpt4v refusing to answer/insisting that there is no image provided when you validate gpt4v with mmmu? I came across the issue above when I tried to send gpt4v the mmmu dataset...

Thanks a lot again!!!!!!!!!!

drogozhang commented 8 months ago

gpt4v is well aligned. So better insist them to provide answers even gpt4v claims it cannot view images.

SweetGUOguo commented 8 months ago

Thank you!!!