search

vllm-project / vllm

A high-throughput and memory-efficient inference and serving engine for LLMs
https://docs.vllm.ai
Apache License 2.0
29.46k stars 4.42k forks source link

[New Model]: DeepSeek VL #4982

Open SinanAkkoyun opened 5 months ago

SinanAkkoyun commented 5 months ago

The model to consider.

https://huggingface.co/deepseek-ai/deepseek-vl-7b-chat

The closest model vllm already supports.

Llava

What's your difficulty of supporting the model you want?

github-actions[bot] commented 6 days ago

This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!

SinanAkkoyun commented 5 days ago

Bump

hmellor commented 4 days ago

I'll mark this as keep-open. Do you plan to update your PR @SinanAkkoyun?

SinanAkkoyun commented 4 days ago

@hmellor Hi! I am more focused on Qwen2-VL (I prematurely commented Bump), but it doesn't work with vLLM sadly, I opened an issue for that

But regarding DeepSeek VL, someone is working on the PR so please keep this open :)

hmellor commented 4 days ago

Ok, no problem! I'll leave the labels as they are

SinanAkkoyun commented 4 days ago

Thank you! ^^