[Usage]: Dose vLLM support embedding api of multimodal llm?

vllm-project / vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

https://docs.vllm.ai

Apache License 2.0

26.88k stars 3.95k forks source link

Open sfyumi opened 4 days ago

sfyumi commented 4 days ago

The output of `python collect_env.py`

eg: get embedding of minicpmv 2.6

[X] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

DarkLight1337 commented 4 days ago

No, this is not supported yet.

DarkLight1337 commented 4 days ago

In fact, this isn't even available for most language-only models. The only one supported right now is Mistral. See also #7915

noooop commented 4 days ago

I am working on it. #8453 #8452 @DarkLight1337