Inference acceleration, can the trained model use some inference framework? The code comes from the llava architecture. Can it be integrated into inference frameworks such as sglang or lmdeploy similar to llava?

BAAI-DCAI / Bunny

A family of lightweight multimodal models.

Apache License 2.0

874 stars 66 forks source link

Inference acceleration, can the trained model use some inference framework? The code comes from the llava architecture. Can it be integrated into inference frameworks such as sglang or lmdeploy similar to llava? #121

Open zhangqingwu opened 3 weeks ago

zhangqingwu commented 3 weeks ago

Inference acceleration, can the trained model use some inference framework? The code comes from the llava architecture. Can it be integrated into inference frameworks such as sglang or lmdeploy similar to llava?