issues
search
EvolvingLMMs-Lab
/
lmms-eval
Accelerating the development of large multimodal models (LMMs) with lmms-eval
https://lmms-lab.github.io/
Other
1.03k
stars
53
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Add task VITATECS
#130
lscpku
opened
21 hours ago
0
add task MMBench-ru
#129
Dannoopsy
opened
1 day ago
0
add task gqa-ru
#128
Dannoopsy
opened
4 days ago
2
Adding MobileCaptureVQA to the benchmark
#127
arnaudstiegler
opened
1 week ago
1
External package integration using plugins
#126
lorenzomammana
closed
1 day ago
7
[Model] aligned llava-interleave model results on video tasks
#125
Luodian
closed
6 days ago
0
[FAQ] about caching the zipped file from HF to local folder
#124
Luodian
opened
1 week ago
0
Propose to print error information when import models
#123
SDaoer
opened
1 week ago
1
[Reproduce] Unable to reproduce AI2D, ChartQA and InfoVQA results for llava-1.6-mistral-7b
#122
GoGoJoestar
opened
1 week ago
0
Cannot find any code / link to worldqa dataset
#121
Hasnat79
opened
1 week ago
1
Add docs for datasets upload to HF
#120
pufanyi
closed
1 week ago
0
Can not load llava_hf models since the new updates!
#119
hasanar1f
opened
1 week ago
4
Fix the potential risk by PR #117
#118
teowu
closed
1 week ago
0
LongVideoBench for LMMs-Eval
#117
teowu
closed
2 weeks ago
0
There is an inconsistency in the number of mmbench images
#116
MuskAI
opened
2 weeks ago
1
Unable to reproduce SQA results for llava-1.5
#115
clairez-cerebras
opened
2 weeks ago
3
add tinyllava
#114
zjysteven
closed
1 week ago
1
Q-Bench, Q-Bench2, A-Bench
#113
teowu
closed
2 weeks ago
0
Error during videomme
#112
yukang2017
opened
2 weeks ago
2
add II-Bench
#111
XinrunDu
closed
2 weeks ago
0
building context takes too long
#110
simplelifetime
opened
2 weeks ago
1
[Small Update] Update the version of LMMs-Eval
#109
pufanyi
closed
2 weeks ago
0
[Upgrade to v0.2] Embracing Video Evaluations with LMMs-Eval
#108
Luodian
closed
2 weeks ago
0
update gpt-3.5-turbo version
#107
AtsuMiyai
closed
2 weeks ago
0
AssertionError: No tasks specified, or no tasks found. Please verify the task names when using local dataset path
#106
MuskAI
opened
2 weeks ago
0
Include VCR
#105
tianyu-z
closed
2 weeks ago
4
Why cot not supported for Mathverse
#104
huiyeruzhou
opened
2 weeks ago
0
Unreasonable data in AI2D dataset used for evaluation
#103
yqy2001
opened
3 weeks ago
3
Allow loading models and tasks from external packages
#102
lorenzomammana
opened
3 weeks ago
1
Update conbench in README
#101
Gumpest
closed
3 weeks ago
0
add Conbench
#100
Gumpest
closed
3 weeks ago
1
How to contribute a new dataset?
#99
JohnTang93
opened
3 weeks ago
1
qwenvl-7b evaluate refcoco|+|g cider and IOU are all None,
#98
AderonHuang
opened
3 weeks ago
14
Add MathVerse in README.md
#97
CaraJ7
closed
3 weeks ago
0
Any plan for supporting Lora peft loading
#96
hxhcreate
opened
4 weeks ago
1
add MM-UPD
#95
AtsuMiyai
closed
3 weeks ago
6
Error when import InstructBLIP
#94
hanqiu-hq
opened
1 month ago
1
Add m3exam
#93
Jiawei-Guo
opened
1 month ago
0
add multi-lingual MMMU tasks
#92
Junpliu
closed
1 month ago
0
how to use “few shot”
#91
woshidengweimo
opened
1 month ago
1
Prompt for MathVista
#90
yqy2001
closed
1 month ago
2
W&B Logging Issue on MMMU & Wrong parsed_pred
#89
AlekseyKorshuk
opened
1 month ago
1
No "llava_llama_3" template
#88
yqy2001
opened
1 month ago
2
Adding microsoft/Phi-3-vision-128k-instruct model.
#87
vfragoso
closed
1 month ago
0
Question on evaluation across multiple tasks
#86
AtsuMiyai
closed
1 month ago
0
how to get three pope results [rad,pop,adv] with lmms-eval?
#85
baiyuting
closed
1 month ago
2
[Week 1] Adding 5 examples for Hindi
#84
simran-khanuja
closed
1 month ago
0
[Feature Request] Evaluating quantized models
#83
zjysteven
closed
1 month ago
3
Error while testing Qwen on POPE
#82
hxhcreate
closed
1 month ago
1
LLaVA Benchmark
#81
yqy2001
opened
1 month ago
2
Next