Failure to reproduce the paper results

LLaVA-VL / LLaVA-NeXT

Apache License 2.0

2.4k stars 167 forks source link

Failure to reproduce the paper results #178

Open yuan-QAQ opened 3 weeks ago

yuan-QAQ commented 3 weeks ago

I cloned the "lmms-lab/LLaVA-NeXT-Interleave-Bench" dataset and "llava-onevision-qwen2-7b-ov" checkpoint from Huggingface to reproduce the results of the paper, but some benchmark results seem to be very different (e.g. IEI, qbench, 3D-Chat, MathVerse, SciVerse). What could be the reason for this?

Luodian commented 3 weeks ago

May I know what's your reproduce pipeline and specific script?