search

livekit / agents

Build real-time multimodal AI applications 🤖🎙️📹
https://docs.livekit.io/agents
Apache License 2.0
4k stars 415 forks source link

google gemini plugin #456

Open makerGeek opened 4 months ago

makerGeek commented 4 months ago

Hi,

I'm trying to build an assistant with google gemini as LLM. Are there any guidelines, docs or examples to follow on how to build a livekit-gemini plugin ?

Thanks

keepingitneil commented 3 months ago

Hey! We'd love this as a contribution. Check out the openai plugin for reference: https://github.com/livekit/agents/blob/main/livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/llm.py

We also have the minimal plugin: https://github.com/livekit/agents/tree/main/livekit-plugins/livekit-plugins-minimal that shows what is needed for the package

makerGeek commented 3 months ago

Thanks! I created a Google llm client inspired by the openai one (Google supports almost the same schema of openai so that wasn't complicated). It works for the chat but not for function calling. Would you like me to make a PR so you can review it and eventually assist me on it?

On Thu, Jul 25, 2024, 21:35 Neil Dwyer @.***> wrote:

Hey! We'd love this as a contribution. Check out the openai plugin for reference: https://github.com/livekit/agents/blob/main/livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/llm.py

We also have the minimal plugin: https://github.com/livekit/agents/tree/main/livekit-plugins/livekit-plugins-minimal that shows what is needed for the package

— Reply to this email directly, view it on GitHub https://github.com/livekit/agents/issues/456#issuecomment-2251261242, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADUJOPWMN74BKFTQVZBY5T3ZOFHR3AVCNFSM6AAAAABLG4QFXWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJRGI3DCMRUGI . You are receiving this because you authored the thread.Message ID: @.***>

theomonnom commented 3 months ago

Hey, yes, please make a PR. We will be happy to help integrate the function-calling stuff!

AphinityApp commented 2 months ago

@makerGeek Were you still planning to make a PR for Google LLM plugin?

If not, then I can work on this and open a PR.

makerGeek commented 2 months ago

Hi Sorry for the delay. I can push the code tomorrow and you can have a look and eventually fix any broken parts. Does that work for you?

AphinityApp commented 2 months ago

Hi Sorry for the delay. I can push the code tomorrow and you can have a look and eventually fix any broken parts. Does that work for you?

Yes, sure no problem. Looking forward to your PR! Thanks