search

firebase / genkit

An open source framework for building AI-powered apps with familiar code-centric patterns. Genkit makes it easy to integrate, test, and deploy sophisticated AI features to Firebase or Google Cloud.
Apache License 2.0
677 stars 91 forks source link

[FR] Support forced tool calling #361

Open lfkellogg opened 4 months ago

lfkellogg commented 4 months ago

Is your feature request related to a problem? Please describe. The Gemini API supports setting a "mode" for function calling: https://ai.google.dev/gemini-api/docs/function-calling#function_calling_mode

For example this allows the developer to choose whether the LLM output should only contain function calls, or if the LLM should decide whether to return text or a function call.

The googleai plugin currently does not support tool_config. This should be added to the custom model config.

Describe the solution you'd like The mode could be specified during the generate() call.

Describe alternatives you've considered N/A

Additional context N/A

MichaelDoyle commented 3 months ago

I'm wondering if we should elevate this to a top level generate/model config. @mbleigh @pavelgj

Open AI, Anthropic, et al support the same:

https://platform.openai.com/docs/api-reference/chat/create#chat-create-tool_choice https://docs.anthropic.com/en/docs/tool-use#forcing-tool-use

Vertex AI:

https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling#tool-config

mbleigh commented 3 months ago

👍 anything shared across 3+ providers feels like it ought to be a built-in. Let's add it to the standard model config and add implementations in our plugins.