search

bentoml / OpenLLM

Run any open-source LLMs, such as Llama 3.1, Gemma, as OpenAI compatible API endpoint in the cloud.
https://bentoml.com
Apache License 2.0
9.69k stars 616 forks source link

feat: Multimodal LLMs? #972

Open Trickster85 opened 4 months ago

Trickster85 commented 4 months ago

Feature request

Is it possible to rum multimodal LLMs like Qwen VL or LLaVa 1.5 using openllm?

Motivation

No response

Other

No response

parano commented 1 month ago

You can use BentoML for those models, which allow user to build custom, multi-modal APIs.

It's possible to support those models in OpenLLM in the future depending on community demand.