search

TinyLLaVA / TinyLLaVA_Factory

A Framework of Small-scale Large Multimodal Models
https://arxiv.org/abs/2402.14289
Apache License 2.0
658 stars 68 forks source link

can not get the same GQA and MME results using the official TinyLLaVA-1.5B weights #48

Closed YFCYFC closed 7 months ago

YFCYFC commented 7 months ago

Hi，brilliant work！And the problem I'm faced with is I can not get the same GQA and MME results using the official TinyLLaVA-1.5B weights from https://huggingface.co/bczhou/TinyLLaVA-1.5B/.I use the official bash scripts to conduct the evaluations with the minor changes that are not relevant to the evaluating logics.There must be something wrong I've done. And the MME result:

image

the GQA result:

image

Thanks a lot!

baichuanzhou commented 7 months ago

Try changing the conv_mode to v1

YFCYFC commented 7 months ago

Try changing the conv_mode to v1 Thank you for your rapid reply.Change 'conv_mode' to 'v1' works for me, and I realize that I took tiny_llama as conv_mode for granted for Tinyllava-1.5B and I missed the important information in readme.For the ones who would encounter the similar problems:

image

Thank you again for the perfect job.