search

Chainlit / chainlit

Build Conversational AI in minutes ⚡️
https://docs.chainlit.io
Apache License 2.0
6.01k stars 768 forks source link

multimodal conversation support #998

Open fire opened 1 month ago

fire commented 1 month ago

Is your feature request related to a problem? Please describe.

A real assistant would not only converse by text but can speak and use video / images.

Describe the solution you'd like A clear and concise description of what you want to happen.

Support text, images, audio and vidoe.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Use OpenAI's chatgpt.

Additional context Add any other context or screenshots about the feature request here

I know that multimodal ais are still a challenge for FOSS tooling.

fire commented 1 month ago

See https://huggingface.co/vonjack/Hermes-2-Pro-BakLLaVA-Mistral-7B

willydouhard commented 1 month ago

Did you check https://github.com/Chainlit/cookbook/tree/main/audio-assistant ?