search

vllm-project / vllm

A high-throughput and memory-efficient inference and serving engine for LLMs
https://docs.vllm.ai
Apache License 2.0
26.61k stars 3.9k forks source link

[New Model]: Google's Paligemma family of models #4833

Closed nfplay closed 2 months ago

nfplay commented 4 months ago

The model to consider.

https://huggingface.co/google/paligemma-3b-pt-896

The closest model vllm already supports.

I think the only visual language model supported right now is LLava but I could be wrong.

What's your difficulty of supporting the model you want?

No response

abrichr commented 4 months ago

MiniCPM is also supported.

Excited to test out how PaliGemma compares, especially when analyzing GUI images: https://github.com/OpenAdaptAI/OpenAdapt/issues/637

ywang96 commented 3 months ago

I'm working on a PR for this currently. See #5189