issues
search
EvolvingLMMs-Lab
/
lmms-eval
Accelerating the development of large multimodal models (LMMs) with lmms-eval
https://lmms-lab.github.io/
Other
1.02k
stars
52
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
"llava_hf" is not registered
#78
kyleliang919
opened
1 month ago
5
Too many requests for MathVista
#77
baichuanzhou
opened
1 month ago
0
Update llava.py - Llama 3 Conversation Messages attribute copied as an immutable tuple
#76
jkitaok
opened
1 month ago
1
An issue during evaluating MathVista
#75
simplelifetime
opened
1 month ago
1
Support for llava-hf/llava-v1.6-x
#74
justinphan3110cais
opened
1 month ago
3
[Feat] Add qwen vl api
#73
kcz358
closed
1 month ago
1
Multinode Evaluation
#72
yqy2001
opened
1 month ago
2
While this lmms-eval support V*Bench?
#71
xmu-xiaoma666
opened
1 month ago
0
Bugfix: WebSRC should be token-level F1 NOT character-level
#70
hunterheiden
closed
1 month ago
0
[New Task] WebSRC (multimodal Q&A on web screenshots)
#69
hunterheiden
closed
1 month ago
1
confusion about llava-next for multi-docvqa
#68
yayafengzi
opened
1 month ago
1
Model will be loaded on different devices when using multiple gpus.
#67
baichuanzhou
closed
1 month ago
6
How to add keywords stopping criterion during evaluation?
#66
baichuanzhou
closed
2 months ago
1
incomplete tasks problem
#65
adrianwestmoon
opened
2 months ago
1
Unreasonably high scores in llava_in_the_wild benchmark
#64
wusize
opened
2 months ago
2
New Task: ScreenSpot - Grounding (REC) and instruction generation (REG) on screens
#63
hunterheiden
closed
2 months ago
2
token: False does not work in gqa.yaml
#62
baiyuting
closed
2 months ago
4
Fix typo in Qwen-VL that was causing "reference before assignment"
#61
tupini07
closed
2 months ago
1
Add MathVerse
#60
CaraJ7
closed
2 months ago
2
add idefics2
#59
jzhang38
closed
1 month ago
1
How to add the new dataset?
#58
rese1f
closed
2 months ago
2
[Benchmarks] RealWorldQA
#57
pufanyi
closed
2 months ago
0
Multilingual LLava bench
#56
gagan3012
closed
1 month ago
1
Fix types to allow nullables in `llava_hf.py`
#55
lewtun
closed
2 months ago
0
add Llava-SGlang
#54
jzhang38
closed
2 months ago
5
is there something wrong with the 000000494869.jpg in lmms-eval MME dataset ?
#53
baiyuting
opened
2 months ago
7
Add REC tasks for testing model ability to locally ground objects, given a description. This adds REC for all RefCOCO datasets.
#52
hunterheiden
closed
2 months ago
4
different result between lmms-eval and llava-eval on text-vqa
#51
yiyexy
opened
2 months ago
4
Issues when running evaluation with multiple processes
#50
simplelifetime
opened
2 months ago
5
OOM issue
#49
yiyexy
closed
2 months ago
1
关于CMMMU生成结果截断的问题
#48
lucasjinreal
opened
2 months ago
4
Add `llava` model for 🤗 Transformers
#47
lewtun
closed
2 months ago
10
[WIP] adding mmbench dev evaluation (#75)
#46
Luodian
closed
2 months ago
2
[WIP] add local evaluation for MMBench dev split
#45
Luodian
closed
2 months ago
0
[WIP] MMBench evaluate dev split locally
#44
Luodian
closed
2 months ago
0
N/A scores when running `mmbench`
#43
lewtun
closed
2 months ago
10
Remove dependency on Llava repo
#42
lewtun
opened
2 months ago
2
Consider using lower bounds on dependencies & using optional dependencies for developer tools
#41
lewtun
opened
2 months ago
3
when we can expect this support for more datasets to be completed?(Raven/NLVR2/IconQA)
#40
littleSunlxy
opened
2 months ago
2
[DO NOT MERGE NOW] [Feat] Few shot eval
#39
pufanyi
opened
2 months ago
0
The discord link is expired
#38
caopulan
closed
2 months ago
2
Joshua/olympiadbench
#37
JvThunder
closed
2 months ago
0
[Fix] repr llava doc
#36
cocoshe
closed
1 month ago
5
Logs saving json enable ensure_assiic False to support better visualize on Chinese
#35
lucasjinreal
opened
3 months ago
0
Llava1.6 Mistral ScienceQA Performance
#34
daniel-z-kaplan
opened
3 months ago
2
ValueError: When passing device_map as a string, the value needs to be a device name (e.g. cpu, cuda:0) or 'auto', 'balanced', 'balanced_low_0', 'sequential' but found .
#33
hasanar1f
closed
2 months ago
1
Can not using multiple batch on MMBench
#32
lucasjinreal
opened
3 months ago
4
Evaluation with multiple GPUs
#31
lucasjinreal
opened
3 months ago
13
03-25 13:08:56 [lmms_eval/models/llava.py:382] ERROR Error LlavaLlamaForCausalLM.forward() got an unexpected keyword argument 'cache_position' in generating
#30
awzhgw
opened
3 months ago
1
Local dataasets config didn't work
#29
lucasjinreal
opened
3 months ago
0
Previous
Next