Remove dependency on Llava repo

[lewtun]

Hello, now that the LLava and LLava-Next models are integrated in transformers, would it be possible to remove the dependency on the Llava repo (https://github.com/haotian-liu/LLaVA)?

This would help with integrating lmms-eval into other codebases where the dependencies of the llava repo can conflict (e.g. because they pin transformers to an old version)

[kcz358]

Hello, I think since the implementation is a bit different for llava-hf and llava, we might need to implement another model for llava-hf. Maybe we will try to include that in the next release

[jzhang38]

@kcz358 I can look into it.