microsoft / autogen

A programming framework for agentic AI. Discord: https://aka.ms/autogen-dc. Roadmap: https://aka.ms/autogen-roadmap
https://microsoft.github.io/autogen/
Creative Commons Attribution 4.0 International
29.89k stars 4.35k forks source link

[.Net]: Document: Improve function call and groupchat document #1922

Open LittleLittleCloud opened 5 months ago

LittleLittleCloud commented 5 months ago

@ekzhu let's use this issue to discuss all the mode FunctionCallMiddleware should support?

What needs to be decided

The current implementation of FunctionCallMiddleware

Overview

FunctionCallMiddleware is used to provide FunctionDefinition and invoke function call to an agent.

Usage

IAgent agent;
var functionCallMiddleware = new FunctionCallMiddleware(
    functions: [getWeatherFunctionDefinition], // provide list of function definitions to an agent to use
    functionMap: { getWeatherFunctionDefinition.Name, getWeatherWrapper } // the function map.
);

The current FunctionCallMiddleware has the following behavoirs according to whether functionMap is provided or not

when functionMap is not provided

In this situation, the agent respond will be returned "as is", even when it's a function call message.

sequenceDiagram
    actor User
    participant FunctionCallMiddleware(innerAgent)
    Note over FunctionCallMiddleware(innerAgent): [GetWeather]
    Note over FunctionCallMiddleware(innerAgent): functionMap: null
    User->>FunctionCallMiddleware(innerAgent): "Hello, what's the weather in Seattle?"
    create actor innerAgent
    FunctionCallMiddleware(innerAgent)->>innerAgent: "Hello, what's the weather in Seattle?"
    destroy innerAgent
    innerAgent->>FunctionCallMiddleware(innerAgent): "{FunctionName: GetWeather}"
    FunctionCallMiddleware(innerAgent)->>User: "{FunctionName: GetWeather}"

When functionMap is provided

If user sends a text message and inner agent returns a function call

In this situation, the FunctionCallMiddleware will invoke that function call and return result as text message as well.

sequenceDiagram
    actor User
    participant FunctionCallMiddleware(innerAgent)
    Note over FunctionCallMiddleware(innerAgent): [GetWeather]
    Note over FunctionCallMiddleware(innerAgent): functionMap: {GetWeatherFunc: lambda...}
    User->>FunctionCallMiddleware(innerAgent): "Hello, what's the weather in Seattle?" : TextMessage
    create actor innerAgent
    FunctionCallMiddleware(innerAgent)->>innerAgent: "Hello, what's the weather in Seattle?"
    destroy innerAgent
    innerAgent->>FunctionCallMiddleware(innerAgent): "{FunctionName: GetWeather}"
    FunctionCallMiddleware(innerAgent)->>FunctionCallMiddleware(innerAgent): Invoke GetWeatherFunc ...
    FunctionCallMiddleware(innerAgent)->>User: "{The result of GetWeatherFunc}" : TextMessage

If user sends a function call message

In this situation, the FunctionCallMiddleware will simply short-circuit the inner agent and return the invoke result as text message.

sequenceDiagram
    actor User
    participant FunctionCallMiddleware(innerAgent)
    Note over FunctionCallMiddleware(innerAgent): [GetWeather]
    Note over FunctionCallMiddleware(innerAgent): functionMap: {GetWeatherFunc: lambda...}
    User->>FunctionCallMiddleware(innerAgent): "{FunctionName: GetWeather}" : ToolCallMessage
    FunctionCallMiddleware(innerAgent)->>FunctionCallMiddleware(innerAgent): Short-curcuit the inner agent and Invoke GetWeatherFunc ...
    FunctionCallMiddleware(innerAgent)->>User: "{The result of GetWeatherFunc}" : ToolCallResultMessage

The new scenario FunctionCallMiddleware needs to support

private function call: inner agent use invoke result to further reply

sequenceDiagram
    actor User
    participant FunctionCallMiddleware(innerAgent)
    Note over FunctionCallMiddleware(innerAgent): [GetWeather]
    Note over FunctionCallMiddleware(innerAgent): functionMap: {GetWeatherFunc: lambda...}
    User->>FunctionCallMiddleware(innerAgent): "Hello, what's the weather in Seattle?" : TextMessage
    create actor innerAgent
    FunctionCallMiddleware(innerAgent)->>innerAgent: "Hello, what's the weather in Seattle?"
    loop: until inner agent reply with a non function call message
    innerAgent->>FunctionCallMiddleware(innerAgent): "{FunctionName: GetWeather}"
    FunctionCallMiddleware(innerAgent)->>innerAgent: "{FunctionCall Result}"
    end
    destroy innerAgent
    innerAgent->>FunctionCallMiddleware(innerAgent): "According to {xxx}, The weather in Seattle is sunny" : TextMessage
    FunctionCallMiddleware(innerAgent)->>User: "According to {xxx}, The weather in Seattle is sunny" : TextMessage

function call with all function call history: invoke the function call and send result back to inner agent to generate further reply, and return all messages.

sequenceDiagram
    actor User
    participant FunctionCallMiddleware(innerAgent)
    Note over FunctionCallMiddleware(innerAgent): [GetWeather]
    Note over FunctionCallMiddleware(innerAgent): functionMap: {GetWeatherFunc: lambda...}
    User->>FunctionCallMiddleware(innerAgent): "Hello, what's the weather in Seattle?" : TextMessage
    create actor innerAgent
    FunctionCallMiddleware(innerAgent)->>innerAgent: "Hello, what's the weather in Seattle?"
    loop: until inner agent reply with a non function call message
    innerAgent->>FunctionCallMiddleware(innerAgent): "{FunctionName: GetWeather}"
    FunctionCallMiddleware(innerAgent)->>innerAgent: "{FunctionCall Result}"
    end
    destroy innerAgent
    innerAgent->>FunctionCallMiddleware(innerAgent): "According to {xxx}, The weather in Seattle is sunny" : TextMessage
    FunctionCallMiddleware(innerAgent)->>User: [...function call history, "According to {xxx}, The weather in Seattle is sunny"] : AggregateMessage
qingyun-wu commented 2 months ago

@LittleLittleCloud is this issue addressed?

LittleLittleCloud commented 2 months ago

Nope, I just put it under 0.0.16 milestone