issues
search
open-compass
/
VLMEvalKit
Open-source evaluation toolkit of large vision-language models (LVLMs), support ~100 VLMs, 40+ benchmarks
https://huggingface.co/spaces/opencompass/open_vlm_leaderboard
Apache License 2.0
1.08k
stars
154
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Qwen2-VL-2B-Instruct与榜单结果对不齐
#492
helloworld01001
opened
16 hours ago
0
MME-RealWorld-CN的评测集适配问题
#491
ManiiXu
opened
21 hours ago
1
[Model] add support for Llama-3.2-11B/90B-Vision-Instruct
#490
FangXinyu-0913
opened
1 day ago
0
[Benchmark] Add MMSearch illustration in README
#489
CaraJ7
opened
1 day ago
0
[Model] add support for XinYuan-VL-2B
#488
thomas-yanxin
opened
2 days ago
1
Fix internvl
#487
vonfeng
closed
3 days ago
0
add BlueLM-V api
#486
rkshuai
closed
1 day ago
0
MMBench_TEST 评估结果是否可以自动提交
#485
Sync-yxh
closed
2 days ago
1
Reproducing QWen2VL Results on Video Benchmarks with VLMEvalKit
#484
aniki-ly
opened
4 days ago
0
[Benchmark] MathVerse
#483
CaraJ7
closed
4 days ago
0
how to run on multi-gpu with device_map='auto'
#482
qianwangn
opened
6 days ago
1
[Models] add moondream1 and moondream2 models
#481
tackhwa
closed
4 days ago
3
liuhaotian/llava-v1.6-vicuna-7b 评测时报错
#480
Cooperx521
opened
1 week ago
1
qwenvl2 run.py 无法一机多卡，每卡一个模型，并行推理一个测评
#479
M3Dade
closed
3 days ago
2
DocVQA测评无法正常使用
#478
M3Dade
closed
3 days ago
1
如何测评大模型的效率
#477
wenyu1009
closed
1 week ago
2
[Model] Add Eagle x series model
#476
tackhwa
closed
1 week ago
0
amber benchmark
#475
yfzhang114
closed
1 week ago
0
[Fix] Update prompts for InternVL2
#474
czczup
closed
1 week ago
0
【提问】关于语音模型的评测
#473
xx0412
opened
1 week ago
1
Add Ovis1.6-Gemma2-9B
#472
runninglsy
closed
1 week ago
0
HallusionBench skips samples without images
#471
ChuanyangZheng
closed
3 days ago
3
[Improvement] Optimize LLaVA-OneVision Inference
#470
kennymckormick
closed
1 week ago
0
infovqa_test测试时会报错
#469
Cooperx521
closed
1 week ago
1
api key是不是在某个地方缓存下来了，当删掉.env不想用gpt评测的时候，还是会报错
#468
Cooperx521
closed
1 week ago
1
'gpt-3.5-turbo-0613'已被废弃，建议将'gpt-3.5-turbo-0613' -> gpt-3.5-turbo
#467
Cooperx521
closed
1 week ago
1
InternVL2 Truncated Output
#466
TJ-Ouyang
closed
3 days ago
4
[Model] Support Pixtral
#465
kennymckormick
closed
1 week ago
0
Fix program error exit without synchronization.
#464
lerogo
closed
1 week ago
0
Fix program error exit without synchronization.
#463
lerogo
closed
1 week ago
0
[Feature]: Add pre-commit.ci integration for automated PR fixes
#462
Mor-Li
closed
1 week ago
0
[Model] fix(qwen2vl) fix ocrbench minpixels
#461
kq-chen
closed
1 week ago
0
qwen2vl_series需要什么版本的transformers
#460
helloworld01001
closed
1 week ago
1
mme-realworld-prompt
#459
yfzhang114
closed
2 weeks ago
0
mme-realworld-prompt
#458
yfzhang114
closed
2 weeks ago
0
support MiniMonkey model
#457
white2018
closed
1 week ago
0
mme-realworld-score
#456
yfzhang114
closed
2 weeks ago
0
Input length of input_ids is 0, but max_length is set to -2009.
#455
wangli68
opened
2 weeks ago
0
评测时PermissionError: [Errno 13] Permission denied
#454
40459447
opened
2 weeks ago
1
测试程序经常中断
#453
HZWHH
closed
2 weeks ago
3
mme-real-world options
#452
yfzhang114
closed
2 weeks ago
3
[Model] add qwen2vl api and fix prompt
#451
kq-chen
closed
2 weeks ago
0
slime
#450
yfzhang114
closed
2 weeks ago
2
MME-RealWorld Not Supported.
#449
Haochen-Wang409
closed
3 weeks ago
2
qwen2-vl-7B
#448
yfzhang114
closed
3 weeks ago
1
[Feature] Support custom_prompt for API models
#447
kennymckormick
closed
3 weeks ago
0
Support TextOCR, DocVQA in GodBench - initial
#446
WuHaohui1231
closed
3 weeks ago
0
[Benchmark] CRPE
#445
ttguoguo3
closed
2 weeks ago
0
Custom prompt for API?
#444
geweihgg
closed
3 weeks ago
1
Update eos_token for HF converted xgen-mm
#443
azshue
closed
3 weeks ago
0
Next