@ekzhu let's use this issue to discuss all the mode FunctionCallMiddleware should support?
What needs to be decided
The behavior of invoking a function call after inner agent generate a function call message, we should provide options to support the following scenarios
private function call: invoke the function and send result back to inner agent to further generate reply but only return the last reply
private function call with all function call history: invoke the function call and send result back to inner agent to generate further reply, and return all messages.
directly return invoke result (The current implementation)
The current implementation of FunctionCallMiddleware
Overview
FunctionCallMiddleware is used to provide FunctionDefinition and invoke function call to an agent.
Usage
IAgent agent;
var functionCallMiddleware = new FunctionCallMiddleware(
functions: [getWeatherFunctionDefinition], // provide list of function definitions to an agent to use
functionMap: { getWeatherFunctionDefinition.Name, getWeatherWrapper } // the function map.
);
The current FunctionCallMiddleware has the following behavoirs according to whether functionMap is provided or not
when functionMap is not provided
In this situation, the agent respond will be returned "as is", even when it's a function call message.
sequenceDiagram
actor User
participant FunctionCallMiddleware(innerAgent)
Note over FunctionCallMiddleware(innerAgent): [GetWeather]
Note over FunctionCallMiddleware(innerAgent): functionMap: null
User->>FunctionCallMiddleware(innerAgent): "Hello, what's the weather in Seattle?"
create actor innerAgent
FunctionCallMiddleware(innerAgent)->>innerAgent: "Hello, what's the weather in Seattle?"
destroy innerAgent
innerAgent->>FunctionCallMiddleware(innerAgent): "{FunctionName: GetWeather}"
FunctionCallMiddleware(innerAgent)->>User: "{FunctionName: GetWeather}"
When functionMap is provided
If user sends a text message and inner agent returns a function call
In this situation, the FunctionCallMiddleware will invoke that function call and return result as text message as well.
sequenceDiagram
actor User
participant FunctionCallMiddleware(innerAgent)
Note over FunctionCallMiddleware(innerAgent): [GetWeather]
Note over FunctionCallMiddleware(innerAgent): functionMap: {GetWeatherFunc: lambda...}
User->>FunctionCallMiddleware(innerAgent): "Hello, what's the weather in Seattle?" : TextMessage
create actor innerAgent
FunctionCallMiddleware(innerAgent)->>innerAgent: "Hello, what's the weather in Seattle?"
destroy innerAgent
innerAgent->>FunctionCallMiddleware(innerAgent): "{FunctionName: GetWeather}"
FunctionCallMiddleware(innerAgent)->>FunctionCallMiddleware(innerAgent): Invoke GetWeatherFunc ...
FunctionCallMiddleware(innerAgent)->>User: "{The result of GetWeatherFunc}" : TextMessage
If user sends a function call message
In this situation, the FunctionCallMiddleware will simply short-circuit the inner agent and return the invoke result as text message.
sequenceDiagram
actor User
participant FunctionCallMiddleware(innerAgent)
Note over FunctionCallMiddleware(innerAgent): [GetWeather]
Note over FunctionCallMiddleware(innerAgent): functionMap: {GetWeatherFunc: lambda...}
User->>FunctionCallMiddleware(innerAgent): "{FunctionName: GetWeather}" : ToolCallMessage
FunctionCallMiddleware(innerAgent)->>FunctionCallMiddleware(innerAgent): Short-curcuit the inner agent and Invoke GetWeatherFunc ...
FunctionCallMiddleware(innerAgent)->>User: "{The result of GetWeatherFunc}" : ToolCallResultMessage
The new scenario FunctionCallMiddleware needs to support
private function call: inner agent use invoke result to further reply
sequenceDiagram
actor User
participant FunctionCallMiddleware(innerAgent)
Note over FunctionCallMiddleware(innerAgent): [GetWeather]
Note over FunctionCallMiddleware(innerAgent): functionMap: {GetWeatherFunc: lambda...}
User->>FunctionCallMiddleware(innerAgent): "Hello, what's the weather in Seattle?" : TextMessage
create actor innerAgent
FunctionCallMiddleware(innerAgent)->>innerAgent: "Hello, what's the weather in Seattle?"
loop: until inner agent reply with a non function call message
innerAgent->>FunctionCallMiddleware(innerAgent): "{FunctionName: GetWeather}"
FunctionCallMiddleware(innerAgent)->>innerAgent: "{FunctionCall Result}"
end
destroy innerAgent
innerAgent->>FunctionCallMiddleware(innerAgent): "According to {xxx}, The weather in Seattle is sunny" : TextMessage
FunctionCallMiddleware(innerAgent)->>User: "According to {xxx}, The weather in Seattle is sunny" : TextMessage
function call with all function call history: invoke the function call and send result back to inner agent to generate further reply, and return all messages.
sequenceDiagram
actor User
participant FunctionCallMiddleware(innerAgent)
Note over FunctionCallMiddleware(innerAgent): [GetWeather]
Note over FunctionCallMiddleware(innerAgent): functionMap: {GetWeatherFunc: lambda...}
User->>FunctionCallMiddleware(innerAgent): "Hello, what's the weather in Seattle?" : TextMessage
create actor innerAgent
FunctionCallMiddleware(innerAgent)->>innerAgent: "Hello, what's the weather in Seattle?"
loop: until inner agent reply with a non function call message
innerAgent->>FunctionCallMiddleware(innerAgent): "{FunctionName: GetWeather}"
FunctionCallMiddleware(innerAgent)->>innerAgent: "{FunctionCall Result}"
end
destroy innerAgent
innerAgent->>FunctionCallMiddleware(innerAgent): "According to {xxx}, The weather in Seattle is sunny" : TextMessage
FunctionCallMiddleware(innerAgent)->>User: [...function call history, "According to {xxx}, The weather in Seattle is sunny"] : AggregateMessage
@ekzhu let's use this issue to discuss all the mode
FunctionCallMiddleware
should support?What needs to be decided
The current implementation of FunctionCallMiddleware
Overview
FunctionCallMiddleware is used to provide
FunctionDefinition
and invoke function call to an agent.Usage
The current FunctionCallMiddleware has the following behavoirs according to whether
functionMap
is provided or notwhen
functionMap
is not providedIn this situation, the agent respond will be returned "as is", even when it's a function call message.
When functionMap is provided
If user sends a text message and inner agent returns a function call
In this situation, the
FunctionCallMiddleware
will invoke that function call and return result as text message as well.If user sends a function call message
In this situation, the
FunctionCallMiddleware
will simply short-circuit the inner agent and return the invoke result as text message.The new scenario
FunctionCallMiddleware
needs to supportprivate function call: inner agent use invoke result to further reply
function call with all function call history: invoke the function call and send result back to inner agent to generate further reply, and return all messages.