search

EvolvingLMMs-Lab / lmms-eval

Accelerating the development of large multimodal models (LMMs) with lmms-eval
https://lmms-lab.github.io/
Other
1.03k stars 54 forks source link

Performance about cuda version #27

Open xmu-xiaoma666 opened 3 months ago

xmu-xiaoma666 commented 3 months ago

I find cuda affects performance significantly.

The cuda is 12.0, the performance on mme is as follow (very low):

image

Then cuda is 12.2, the performance on mme is as follow:

image

If I only have a machine with cuda 12.0, and I can't upgrade the Cuda version due to gpu driver. How can I reproduce the performance?

kcz358 commented 3 months ago

Hi @xmu-xiaoma666 , maybe you can try using torch version 2.0.1 and cuda 11.7 version which is used in llava1.5