search

microsoft / semantic-kernel

Integrate cutting-edge LLM technology quickly and easily into your apps
https://aka.ms/semantic-kernel
MIT License
21.39k stars 3.15k forks source link

.Net: Drop duplicated function calling metadata #6872

Open markwallace-microsoft opened 3 months ago

markwallace-microsoft commented 3 months ago

We are currently duplicating storing function calling metadata to support backward compatibility. We will drop the old format as part of the breaking changes required for the OpenAI client library upgrade.

Context Today, SK keeps almost the same information about function calls (functions LLM requested to call) in three different places within the same chat message:

  1. In the OpenAIChatMessageContent.ToolCalls property.
  2. As items in the OpenAIChatMessageContent.Metadata dictionary with the ChatResponseMessage.FunctionToolCalls key.
  3. As items of the FunctionCallContent type in the ChatMessageContent.Items collection.

The first place the ToolCalls property is used is to get access to the function calls and break-glass scenarios. The function calls are represented by data model types that are specific to a particular connector which means that the function calls can't be shared between connectors.

The second place is only needed and used to preserve function call information in scenarios where chat history is serialized and deserialized later.

The third place, Items, was recently added as storage for the function call information. The function call information is stored in the form of items of the FunctionCallContent type, which is designed to be connector-agnostic, serializable and represent function calls universally, regardless of the LLM requested to call those functions. More details can be found here - 0041-function-call-content.md.

ToDo Taking into account that the function call information represented by the FunctionCallContent type is sufficient to cover the serialization/deserialization scenarios and provides a connector-agnostic way to access the information, it makes sense to keep it and deprecate functionality that stores the duplicating information in the OpenAIChatMessageContent.Metadata dictionary to minimize the size of the serialized chat history and simplify code on the AI connector's side.

SergeyMenshykh commented 1 week ago

This issue should be considered as part of the obsolescence of the current function-calling model when the new one is implemented.