search

BradyFU / Awesome-Multimodal-Large-Language-Models

:sparkles::sparkles:Latest Advances on Multimodal Large Language Models
12.45k stars 795 forks source link

Update Qwen-VL results #56

Closed jinze1994 closed 1 year ago

jinze1994 commented 1 year ago

We are honored to evaluate the Qwen-VL series on your good work MME Benchmark.

Qwen-VL-Chat achieved the SOTAs on MME until now. We provide all code and steps HERE to reproduce the results.

We would appreciate it if you update these changes on your home page and pictures as soon as possible.

=========== Perception ===========
total score: 1487.576330532213 

         existence  score: 158.33333333333331
         count  score: 150.0
         position  score: 128.33333333333334
         color  score: 170.0
         posters  score: 178.57142857142856
         celebrity  score: 120.58823529411764
         scene  score: 152.25
         landmark  score: 164.0
         artwork  score: 125.5
         OCR  score: 140.0

=========== Cognition ===========
total score: 360.71428571428567 

         commonsense_reasoning  score: 130.7142857142857
         numerical_calculation  score: 40.0
         text_translation  score: 147.5
         code_reasoning  score: 42.5
BradyFU commented 1 year ago

Thanks! The results of Qwen-VL have been updated : )