search

bentoml / OpenLLM

Run any open-source LLMs, such as Llama, Mistral, as OpenAI compatible API endpoint in the cloud.
https://bentoml.com
Apache License 2.0
10.12k stars 641 forks source link

feat: Multimodal LLMs? #972

Open Trickster85 opened 7 months ago

Trickster85 commented 7 months ago

Feature request

Is it possible to rum multimodal LLMs like Qwen VL or LLaVa 1.5 using openllm?

Motivation

No response

Other

No response

parano commented 4 months ago

You can use BentoML for those models, which allow user to build custom, multi-modal APIs.

It's possible to support those models in OpenLLM in the future depending on community demand.