Update for Otter-Image-MPT7B and Otter-Video

AILab-CVC / SEED-Bench

(CVPR2024)A benchmark for evaluating Multimodal LLMs using multiple-choice questions.

Other

276 stars 8 forks source link

Update for Otter-Image-MPT7B and Otter-Video #1

Open Luodian opened 11 months ago

Luodian commented 11 months ago

Thanks for the wonderful evaluation work! But the Otter evaluation seems based on our early version (around May 2023), we have a stronger MPT7B version Otter since last month. Would you like to try to evaluate it instead of the LLama7B version.

We also have a video version of Otter, would you like to add it to VideoLLM evaluation?

Otter-Image: https://huggingface.co/luodian/OTTER-Image-MPT7B Otter-Video: https://huggingface.co/luodian/OTTER-Video-LLaMA7B-DenseCaption

geyuying commented 10 months ago

Thank you for your attention to our SEED-Bench.

We have released SEED-Bench leaderboard in https://huggingface.co/spaces/AILab-CVC/SEED-Bench_Leaderboard and you can update the results of your models in the leaderboard by following our evaluation instructions.