search

deepseek-ai / DeepSeek-VL

DeepSeek-VL: Towards Real-World Vision-Language Understanding
https://huggingface.co/spaces/deepseek-ai/DeepSeek-VL-7B
MIT License
2.03k stars 190 forks source link

How to finetune and serve the finetuned model using VLLM like tools #2

Open monksgoyal opened 6 months ago

monksgoyal commented 6 months ago

Hi I want to finetune it on my custom OCR tasks, can you share finetuning script and also help in telling the best way to serve it very very fast.

xs818818 commented 6 months ago

同问

brucethemoose commented 6 months ago

SGlang support would be excellent, as it doesn't seem to work out of the box.

brucethemoose commented 6 months ago

Made an issue for it here as well: https://github.com/sgl-project/sglang/issues/297

SGlang already supports llava, so perhaps adding deepseek would not be as huge of a jump as vllm (which currently does not support vision models).