microsoft / autogen

A programming framework for agentic AI 🤖
https://microsoft.github.io/autogen/
Creative Commons Attribution 4.0 International
32.64k stars 4.75k forks source link

[Bug]: tool response will be wrong when tool_caller‘s llm_config include stream=True and n > 0 #3506

Open yinzuoweia opened 1 month ago

yinzuoweia commented 1 month ago

Describe the bug

when set stream=True and n > 0 in llm_config. OpenAIClient.create will raise an exception cause type != "function" The debug variables like:

CleanShot 2024-09-11 at 17 33 23

Steps to reproduce

import os
from autogen import ConversableAgent, UserProxyAgent

tool_caller = ConversableAgent(
    name="tool_caller",
    llm_config={
        "config_list": [{"model": "gpt-4o", "api_key": os.getenv("OPENAI_API_KEY"), "cache_seed": None, "n": 3, "stream": True}],
    },
    human_input_mode="NEVER",
    system_message="You are a calculator. Use your tools to solve tasks for users.",
)

user_proxy = UserProxyAgent(
    name="user",
    system_message="a human admin",
    human_input_mode="NEVER",
    max_consecutive_auto_reply=0,
    code_execution_config=False
)

def add(a: int , b: int) -> int:
    """
    add two numbers
    """
    return a + b

user_proxy.register_for_execution(name="add")(add)
tool_caller.register_for_llm(name="add", description="add two numbers")(add)

user_proxy.send("what is 1 + 2?", tool_caller, request_reply=True)
chat_messages = tool_caller.chat_messages[user_proxy]

print(1)

Model Used

gpt-4o

Expected Behavior

every choice should has it's own tool call response

Screenshots and logs

No response

Additional Information

No response

yinzuoweia commented 1 month ago

CleanShot 2024-09-12 at 10 30 25

Maybe you should use choice.index instead of tool_calls_chunk.index