issues
search
open-compass
/
VLMEvalKit
Open-source evaluation toolkit of large vision-language models (LVLMs), support 160+ VLMs, 50+ benchmarks
https://huggingface.co/spaces/opencompass/open_vlm_leaderboard
Apache License 2.0
1.34k
stars
188
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
XComposer cannot evaluate on Bench1_TEST
#507
yansuoyuli
closed
1 month ago
4
Fix Y/N type error of POPE
#506
hhaAndroid
closed
1 month ago
0
无法评测llava1.5-7b
#505
itsqyh
closed
1 week ago
4
DocVQA_TEST和InfoVQA_TEST无法评测
#504
helloworld01001
closed
3 weeks ago
1
The evaluation result were all wrong after upgrade
#503
MonolithFoundation
closed
1 month ago
5
AttributeError: 'TSVDataset' object has no attribute 'MODALITY'
#502
MonolithFoundation
closed
1 month ago
1
[Feature]: Add POINTS
#501
YuanLiuuuuuu
closed
1 month ago
0
Adding Pixtral from Mistral Team
#500
amitbcp
closed
1 month ago
2
在MCQ任务中，一道题目具有多张图片，我应该如何构建框架需要的tsv数据集？
#499
Nefefilibata
closed
3 weeks ago
1
add GMAI_MMBench Test
#498
TousenKaname
closed
1 month ago
0
textonly benchmarks
#497
JcWang20
opened
1 month ago
1
Feature add bailingmm
#496
ChuanyangZheng
closed
3 weeks ago
1
ValueError: Unrecognized configuration class <class 'transformers.models.qwen2_vl.configuration_qwen2_vl.Qwen2VLConfig'> for this kind of AutoModel: AutoModelForCausalLM.
#495
LinguaLogician
closed
1 month ago
2
[Model] add kosmos2
#494
tackhwa
closed
1 month ago
4
[Help Wanted] the alignment with official accuracy in llama3.2-vision
#493
droidXrobot
opened
1 month ago
8
Qwen2-VL-2B-Instruct与榜单结果对不齐
#492
helloworld01001
closed
1 month ago
2
MME-RealWorld-CN的评测集适配问题
#491
ManiiXu
closed
1 month ago
1
[Model] add support for Llama-3.2-11B/90B-Vision-Instruct
#490
FangXinyu-0913
closed
1 month ago
2
[Benchmark] Add MMSearch illustration in README
#489
CaraJ7
closed
1 month ago
0
[Model] add support for XinYuan-VL-2B
#488
thomas-yanxin
closed
1 month ago
2
Fix internvl
#487
vonfeng
closed
1 month ago
0
add BlueLM-V api
#486
rkshuai
closed
1 month ago
0
MMBench_TEST 评估结果是否可以自动提交
#485
Sync-yxh
closed
1 month ago
1
Reproducing QWen2VL Results on Video Benchmarks with VLMEvalKit
#484
aniki-ly
opened
1 month ago
4
[Benchmark] MathVerse
#483
CaraJ7
closed
1 month ago
0
how to run on multi-gpu with device_map='auto'
#482
qianwangn
closed
1 month ago
3
[Models] add moondream1 and moondream2 models
#481
tackhwa
closed
1 month ago
3
liuhaotian/llava-v1.6-vicuna-7b 评测时报错
#480
Cooperx521
closed
1 month ago
1
qwenvl2 run.py 无法一机多卡，每卡一个模型，并行推理一个测评
#479
M3Dade
closed
1 month ago
2
DocVQA测评无法正常使用
#478
M3Dade
closed
1 month ago
1
如何测评大模型的效率
#477
wenyu1009
closed
1 month ago
2
[Model] Add Eagle x series model
#476
tackhwa
closed
1 month ago
0
amber benchmark
#475
yfzhang114
closed
1 month ago
1
[Fix] Update prompts for InternVL2
#474
czczup
closed
1 month ago
0
【提问】关于语音模型的评测
#473
xx0412
opened
1 month ago
1
Add Ovis1.6-Gemma2-9B
#472
runninglsy
closed
1 month ago
0
HallusionBench skips samples without images
#471
ChuanyangZheng
closed
1 month ago
3
[Improvement] Optimize LLaVA-OneVision Inference
#470
kennymckormick
closed
1 month ago
0
infovqa_test测试时会报错
#469
Cooperx521
closed
1 month ago
1
api key是不是在某个地方缓存下来了，当删掉.env不想用gpt评测的时候，还是会报错
#468
Cooperx521
closed
1 month ago
1
'gpt-3.5-turbo-0613'已被废弃，建议将'gpt-3.5-turbo-0613' -> gpt-3.5-turbo
#467
Cooperx521
closed
1 month ago
1
InternVL2 Truncated Output
#466
TJ-Ouyang
closed
1 month ago
4
[Model] Support Pixtral
#465
kennymckormick
closed
2 months ago
0
Fix program error exit without synchronization.
#464
lerogo
closed
2 months ago
3
Fix program error exit without synchronization.
#463
lerogo
closed
2 months ago
0
[Feature]: Add pre-commit.ci integration for automated PR fixes
#462
Mor-Li
closed
2 months ago
0
[Model] fix(qwen2vl) fix ocrbench minpixels
#461
kq-chen
closed
2 months ago
0
qwen2vl_series需要什么版本的transformers
#460
helloworld01001
closed
2 months ago
3
mme-realworld-prompt
#459
yfzhang114
closed
2 months ago
0
mme-realworld-prompt
#458
yfzhang114
closed
2 months ago
0
Previous
Next