Feature Request: Support for Qwen2-VL

[isr431]

Prerequisites

• [X] I am running the latest code. Mention the version if possible as well.
• [X] I carefully followed the README.md.
• [X] I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• [X] I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

Qwen just released Qwen2-VL 2B & 7B under the Apache 2.0 License.

Motivation

SoTA understanding of images of various resolution & ratio: Qwen2-VL achieves state-of-the-art performance on visual understanding benchmarks, including MathVista, DocVQA, RealWorldQA, MTVQA, etc. Understanding videos of 20min+: Qwen2-VL can understand videos over 20 minutes for high-quality video-based question answering, dialog, content creation, etc.

Possible Implementation

No response

[chigkim]

+1 This would be another great addition!

[crzroot]

This model is awesome

[suepradun]

I am looking forward to it very much

[xzlinux]

+1 I am looking forward to it very much

[yukiarimo]

We can try Llamafing it

[XDesktopSoft]

+1

[WildCatApp]

+1

[uestcbraid]

+1

[mrhalyang]

+1

[elyzionz]

+1

[2OsZI4ISYd]

+1

[Kimizhao]

+1

[enryteam]

+1

[yukiarimo]

Any updates?

[apipino]

+1

[Xhehab]

+1

[Seaman3body]

+1

[zenoverflow]

+1

[whoisltd]

+1

[eav-solution]

+1

[feynmanloo]

I can not wait for it !!!

[chigkim]

Maybe people should also express interest and ask Qwen2-VL devs to implement. https://github.com/QwenLM/Qwen2-VL/issues/7

[wmx-github]

Expect to use llama.cpp end side inference

[HimariO]

Is anyone already working on this? If not, I would like to give it a try.

[solangii]

+1 is there any updates?

[PredyDaddy]

+1

[shobhit9618]

+1

[zhouxihong1]

+1