search

open-compass / VLMEvalKit

Open-source evaluation toolkit of large vision-language models (LVLMs), support 160+ VLMs, 50+ benchmarks
https://huggingface.co/spaces/opencompass/open_vlm_leaderboard
Apache License 2.0
1.34k stars 188 forks source link

create new benchmark #409

Closed yansuoyuli closed 2 months ago

yansuoyuli commented 2 months ago

您好，我想请问一下，如何把数据集变成这个框架下的.tsv文件呀？

BrenchCC commented 2 months ago

你这个问题问的不是很清楚，你可以根据数据集的类型例如MCQ，VQA，yes or no类型的问题去下载该框架下对应的tsv文件，在'''vlmeval/dataset’‘’路径下的脚本中记录着tsv的下载链接，参考相关的文件创建你自己的数据集的tsv文件以及相关处理接入代码

kennymckormick commented 2 months ago

Hi, @yansuoyuli , 可以参照我们的开发指南：https://github.com/open-compass/VLMEvalKit/blob/main/docs/zh-CN/advanced_guides/Development.md