[Fix] Align LLAVA-OneVision Model Loading with Official Implementation

open-compass / VLMEvalKit

Open-source evaluation toolkit of large vision-language models (LVLMs), support 160+ VLMs, 50+ benchmarks

https://huggingface.co/spaces/opencompass/open_vlm_leaderboard

Apache License 2.0

1.34k stars 188 forks source link

[Fix] Align LLAVA-OneVision Model Loading with Official Implementation #432

Closed Mor-Li closed 2 months ago

Mor-Li commented 2 months ago

This pull request updates the model loading process in the llava-onevision implementation to match the official method provided by Hugging Face. The key changes include setting the device_map to 'auto' during model loading and removing the manual tying of model weights. This ensures compatibility and consistency with the official loading procedure, as outlined here.

kennymckormick commented 2 months ago

Close for now. Feel free to reopen when the PR is ready.