search

shikras / shikra

Other
710 stars 44 forks source link

Inconsistent performance on MMBench #44

Open scenarios opened 10 months ago

scenarios commented 10 months ago

I evaluate the released model with your official inference code on MMBench but only got 36.x overall accuracy. It does not match the performance on the leaderboard.

Do you have any suggestion?

yhcao6 commented 9 months ago

Sir, I tested Shikra on MMBench dev and got 39.x overall accuracy. Do you have any updates on the progress?