search

EvolvingLMMs-Lab / lmms-eval

Accelerating the development of large multimodal models (LMMs) with lmms-eval
https://lmms-lab.github.io/
Other
1.33k stars 92 forks source link

[ERROR] Failed to run the `mathvista_test` tasks #189

Open liguopeng0923 opened 1 month ago

liguopeng0923 commented 1 month ago
image

After I add the following code to tasks/mathvista/mathvista_test.yaml, the code runs successfully. But it get an error result.

shot: 0
use_caption: False
use_ocr: False
image
Luodian commented 1 month ago

I think you can try mathvista_testmini, it works on my side.

liguopeng0923 commented 1 month ago

Hi,

Thanks for your timely reply. mathvista_testmini works on my side too, but my baselines report the results of mathvista_test😭.

Luodian commented 1 month ago

oh okay, can you try to solve this issue and welcome a new PR, thanks!

liguopeng0923 commented 1 month ago

ok, I will try it today