vidas
commented
3 months ago tool_choice options "none" and an object requiring specific functions are supported (assuming the model is trained for tool calling and has corresponding chat format).
tool_choice options "auto", and "required" are only supported for select models with known chat formats (as it requires custom altered prompts and custom response parsing).
In general, these two options are not supported, but can be added to work with specific models.
User Case: The user wants to control the model’s behavior regarding tool usage. By specifying tool_choice, the user can determine whether the model should refrain from calling tools, optionally call them, or be required to call them. They can also force the model to use a specific tool by providing its details.
OpenAI API Reference: Chat Completion Docs - tool_choice
Detailed Description of Parameters: • Type: string or object • Optional: Yes
Description: Determines how the model chooses whether to call tools or not, and which tool to call if specified. If tool_choice is a string: • "none": The model will not call any tool and will produce a message instead. • "auto": The model is free to either produce a message or call one or more tools as it sees fit. • "required": The model must call one or more tools and cannot simply produce a message without tool usage. If tool_choice is an object: This is used to force the model to call a specific tool (currently, only functions are supported as tools). • The object must be of the form:
{ "type": "function", "function": { "name": ""
}
}
Here, must match one of the tools provided in the tools parameter.
Behavior: • If no tools are provided, tool_choice defaults to "none". • If tools are provided and tool_choice is not specified, it defaults to "auto". • Setting tool_choice: "none" means the model will not use any tools and will just generate a message. • Setting tool_choice: "auto" allows the model to decide whether to call tools or just produce a response. • Setting tool_choice: "required" means the model must call at least one tool; it cannot return a message without using a tool. • Setting tool_choice to an object specifying a particular tool forces the model to call that tool, overriding any automatic decision-making.
Examples:
{ "tool_choice": "none" }
The model will not call any tools (assuming none are provided) and will just produce a text message.
{ "tool_choice": "auto" }
The model can choose to call a tool or produce a normal response.
{ "tool_choice": { "type": "function", "function": { "name": "calculate_sum" } } }
The model must call the calculate_sum function and produce a function_call response, regardless of context.
"tool_choice": "required"
The model must call at least one tool and cannot simply provide a text response.