Is there any special handling for improving the MME scores

BradyFU / Woodpecker

✨✨Woodpecker: Hallucination Correction for Multimodal Large Language Models. The first work to correct hallucinations in MLLMs.

602 stars 29 forks source link

Is there any special handling for improving the MME scores #9

Closed rubylan closed 10 months ago

rubylan commented 10 months ago

Hi there,

Great work! I am wondering whether the woodpecker can stably raise the MME scores since I found that some sub-tasks (e.g., position, color) did not perform well after the correction.

Details: Following the paper, I've changed the MLLM model output to the format of <yes/no + question (like 'there is xx').>

Is there anything I've missed that caused inferior results?

Thanks a lot!

BradyFU commented 10 months ago

Would you mind leave your WeChat ID? We can communicate further through WeChat.