issues
search
open-compass
/
VLMEvalKit
Open-source evaluation toolkit of large vision-language models (LVLMs), support 160+ VLMs, 50+ benchmarks
https://huggingface.co/spaces/opencompass/open_vlm_leaderboard
Apache License 2.0
1.39k
stars
194
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
[Feat] add modelscope download video datasets
#623
Yunnglin
opened
19 hours ago
0
[Model] Add new model: Prism
#622
Myhs-phz
opened
20 hours ago
0
two bugs 1. I can not use quick start from the README.md . 2 'vlmutil check Llama-3.2-11B-Vision-Instruct ' will lead to error which is 'File "path/VLMEvalKit/vlmeval/dataset/__init__.py", line 168, in DATASET_TYPE if 'openended' in dataset.lower(): AttributeError: 'NoneType' object has no attribute 'lower''
#621
GoogleAlphaZero
opened
20 hours ago
0
[Model] Add TeleMM
#620
CMeteor
opened
21 hours ago
0
[Benchmark] Support DynaMath
#619
kennymckormick
closed
1 day ago
0
[Benchmark] Support MM-Math
#618
kennymckormick
closed
2 days ago
0
[Model] Add LLaVA-OneVision HF
#617
zsoltzsolt
closed
1 day ago
0
[Model] Add LLaVA-OneVision HF
#616
zsoltzsolt
closed
2 days ago
0
[Model] Support SmolVLM
#615
mfarre
closed
2 days ago
1
[Benchmark] Added MMGenBench benchmark
#614
lerogo
closed
21 hours ago
11
[Fix] Enable --reuse with resume from original pkl files with same commit id
#613
FangXinyu-0913
closed
2 days ago
0
Can not download wildvision dataset from the default link
#612
czczup
closed
3 days ago
2
Where is the Mathverse result？
#611
StarUniversus
opened
3 days ago
0
[Improvement] Launch Evaluation w. Config
#610
kennymckormick
closed
1 day ago
0
[Benchmark] Add new benchmark: VizWiz
#609
Myhs-phz
opened
4 days ago
1
[Dataset] Support WildVision Bench
#608
kennymckormick
closed
6 days ago
0
llava-onevision multi-image
#607
jun0wanan
opened
1 week ago
1
add benchmark mme-realworld-lite
#606
yfzhang114
closed
6 days ago
0
Error: module 'torch.library' has no attribute 'register_fake'
#605
tjasmin111
opened
1 week ago
0
[Fix Minor] Fix DATASET_MODALITY
#604
SYuan03
closed
1 week ago
0
[Benchmark] Add new benchmark: Olympiadbench
#603
Myhs-phz
closed
6 days ago
0
[Benchmark] Fix MIA-Bench
#602
Myhs-phz
closed
1 week ago
0
update InternVL2-8B-MPO config
#601
Weiyun1025
closed
1 week ago
0
[Benchmark] Add new benchmark: OlympiadBench
#600
Myhs-phz
closed
1 week ago
0
[Fix] Fix preproc w. role
#599
kennymckormick
closed
1 week ago
0
about Qwen2-vl
#598
jun0wanan
closed
1 week ago
3
[Benchmark] Add new benchmark: Mia-Bench
#597
Myhs-phz
closed
1 week ago
0
[Benchmark] Add New Datasets: Mia-Bench
#596
Myhs-phz
closed
1 week ago
0
Ovis1.5-Llama3-8B在Hallusion Bench上的指标和榜单上的指标差距过大
#595
LIRENDA621
opened
1 week ago
1
Jtvlm branch
#594
jiutiancv
closed
1 week ago
0
fixed MVBench preprocessing
#593
srikant86panda
closed
1 week ago
1
Feat Request: Support for LLAVA OneVision HuggingFace Format Models
#592
zjwu0522
closed
1 day ago
0
评测InternVL2-1B报错： got multiple values for keyword argument 'return_dict'
#591
qingchen177
opened
1 week ago
3
How soon is the 'News' section in the README updated after a new Benchmark or Model is added?
#590
Baiqi-Li
opened
1 week ago
1
MMBench-Video Dataset Download Bug
#589
Noctis-SC
closed
1 week ago
4
Evaluations get tuck on the last 4 questions
#588
XuGW-Kevin
opened
1 week ago
3
[Model] Add RBDash v1.5
#587
anzhao920
closed
6 days ago
0
Update config.py
#586
anzhao920
closed
2 weeks ago
0
AZURE_OPENAI需要怎么配置
#585
helloworld01001
closed
2 weeks ago
0
ChartQA eval bug
#584
lemonliu1992
opened
2 weeks ago
0
[Improvement] Save evaluation results w. git commit ID and evaluation date, to improve reproducibility
#583
kennymckormick
closed
2 weeks ago
0
[Add] Benchmark: NaturalBench (NeurIPS24)
#582
Baiqi-Li
closed
2 weeks ago
3
Vila word
#581
Tianhui-Liu
closed
2 weeks ago
0
Reproducing Qwen2-VL-72B-Instruct Evaluation Results Fails
#580
ChuanyangZheng
closed
1 day ago
5
[Doc] Update README & Quickstart
#579
kennymckormick
closed
2 weeks ago
0
Corrupted MathVista testmini data
#578
LeoDu0314
opened
2 weeks ago
1
AI2d gpt和claude3.5官方分数非常高
#577
Violettttee
opened
2 weeks ago
1
Add Vintern-1B-v2
#576
huynhbaobk
closed
1 week ago
0
update the amber.tsv file and md5 of amber.tsv
#575
yfzhang114
closed
2 weeks ago
0
update CoT prompt for internvl
#574
Weiyun1025
closed
2 weeks ago
0
Next