As of Nov 6, 2023, OpenAI added the ability for a single assistant message to request calling multiple functions in
parallel, and wrapped all function calls in a ToolCall wrapper. In order to add support for this in kani while
maintaining backwards compatibility with OSS function calling models, a ChatMessage now actually maintains the
following internal representation:
ChatMessage.function_call is actually an alias for ChatMessage.tool_calls[0].function. If there is more
than one tool call in the message, when trying to access this property, kani will raise an exception.
To translate kani's FUNCTION message types to OpenAI's TOOL message types, the OpenAIEngine now performs a translation based on binding free tool call IDs to following FUNCTION messages deterministically.
Breaking Changes
To the kani end user, there should be no change to how functions are defined and called. One breaking change was necessary:
Kani.do_function_call and Kani.handle_function_call_exception now take an additional tool_call_id parameter, which may break overriding functions. The documentation has been updated to encourage overriders to handle *args, **kwargs to prevent this happening again.
New Features
kani can now handle making multiple function calls in parallel if the model requests it. Rather than returning an ASSISTANT message with a single function_call, an engine can now return a list of tool_calls. kani will resolve these tool calls in parallel using asyncio, and add their results to the chat history in the order of the list provided.
Returning a single function_call will continue to work for backwards compatibility.
As of Nov 6, 2023, OpenAI added the ability for a single assistant message to request calling multiple functions in parallel, and wrapped all function calls in a
ToolCallwrapper. In order to add support for this in kani while maintaining backwards compatibility with OSS function calling models, aChatMessagenow actually maintains the following internal representation:ChatMessage.function_callis actually an alias forChatMessage.tool_calls[0].function. If there is more than one tool call in the message, when trying to access this property, kani will raise an exception.To translate kani's FUNCTION message types to OpenAI's TOOL message types, the OpenAIEngine now performs a translation based on binding free tool call IDs to following FUNCTION messages deterministically.
Breaking Changes
To the kani end user, there should be no change to how functions are defined and called. One breaking change was necessary:
Kani.do_function_callandKani.handle_function_call_exceptionnow take an additionaltool_call_idparameter, which may break overriding functions. The documentation has been updated to encourage overriders to handle*args, **kwargsto prevent this happening again.New Features
kani can now handle making multiple function calls in parallel if the model requests it. Rather than returning an ASSISTANT message with a single
function_call, an engine can now return a list oftool_calls. kani will resolve these tool calls in parallel using asyncio, and add their results to the chat history in the order of the list provided.Returning a single
function_callwill continue to work for backwards compatibility.