[model] support vision language model llava.

vectorch-ai / ScaleLLM

A high-performance inference system for large language models, designed for production environments.

https://docs.vectorch.com/

Apache License 2.0

317 stars 24 forks source link

[model] support vision language model llava. #178

Closed liutongxuan closed 1 week ago

guocuimi commented 1 week ago

LGTM. Thank you for your hard work in getting the first VLM up and running. I've left some comments; I believe we should refactor to reduce redundant code between the worker and engine after landing this.