Open SinanAkkoyun opened 4 months ago
Hi! Thank you for your amazing work. I was wondering if DeepSeek-VL support would be possible? I'd love to serve a gemv_fast quant of it
gemv_fast
(I think it is llama2 + the two vision encoders, but I believe one would copy HF inference code like for Llava)
I'd love to help if needed. I never implemented a model into an inference engine, if you are too busy to implement it yourself, could you please outline what I need to do?
Would you be interested?
Hi! Thank you for your amazing work. I was wondering if DeepSeek-VL support would be possible? I'd love to serve a
gemv_fast
quant of it(I think it is llama2 + the two vision encoders, but I believe one would copy HF inference code like for Llava)