modelscope / agentscope

Start building LLM-empowered multi-agent applications in an easier way.
https://doc.agentscope.io/
Apache License 2.0
4.81k stars 294 forks source link

litellm_model vision support #395

Open zyzhang1130 opened 1 month ago

zyzhang1130 commented 1 month ago

name: Pull Request about: Create a pull request

Description

Add vision modality support for some of the models supported by LiteLLM. For the LLMs that have vision modalities and support using URL to send images on LiteLLM platform, simply add URL of the image in the 'url' field in the relevant message to pass the image to the LLM API. (see docstring for an example) reference: https://docs.litellm.ai/docs/completion/vision#checking-if-a-model-supports-vision

Checklist

Please check the following items before code is ready to be reviewed.