difference between "lmms-lab/MMBench" in huggingface

open-compass / MMBench

Official Repo of "MMBench: Is Your Multi-modal Model an All-around Player?"

Apache License 2.0

152 stars 10 forks source link

Closed Coobiw closed 5 months ago

Coobiw commented 5 months ago

Hello! I find it's accessible for me to use huggingface to get MMBench(lmms-lab/MMBench). For example:

from datasets import load_dataset
dataset = load_dataset(f"{my_path}/lmms-lab/MMBench/en")['validation'] # ['test']

Then, I get the EN val/test split data. Is there any difference between this repo/VLMEvalKit and lmms-lab/MMBench? Thanks for your reply!

kennymckormick commented 5 months ago

Hi, @Coobiw , I'm not sure, that data file is not served my VLMEvalKit thus I can not provide any guarantee.

Coobiw commented 5 months ago

thanks！i will double check it！