search

microsoft / autogen

A programming framework for agentic AI 🤖
https://microsoft.github.io/autogen/
Creative Commons Attribution 4.0 International
31.73k stars 4.62k forks source link

OpenAI Error code 400 intermitently in groupchats #3679

Open Nathan-Intergral opened 1 day ago

Nathan-Intergral commented 1 day ago

What happened?

I have been experiencing intermittent errors in my group chat executions returning the following error:

 File "/home/nathan/miniconda3/envs/opspilot-query-auto-agent/lib/python3.11/site-packages/autogen/agentchat/conversable_agent.py", line 1180, in a_initiate_chat
    await self.a_send(msg2send, recipient, silent=silent)
  File "/home/nathan/miniconda3/envs/opspilot-query-auto-agent/lib/python3.11/site-packages/autogen/agentchat/conversable_agent.py", line 800, in a_send
    await recipient.a_receive(message, self, request_reply, silent)
  File "/home/nathan/miniconda3/envs/opspilot-query-auto-agent/lib/python3.11/site-packages/autogen/agentchat/conversable_agent.py", line 951, in a_receive
    reply = await self.a_generate_reply(sender=sender)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nathan/miniconda3/envs/opspilot-query-auto-agent/lib/python3.11/site-packages/autogen/agentchat/conversable_agent.py", line 2138, in a_generate_reply
    final, reply = await reply_func(
                   ^^^^^^^^^^^^^^^^^
  File "/home/nathan/miniconda3/envs/opspilot-query-auto-agent/lib/python3.11/site-packages/autogen/agentchat/groupchat.py", line 1214, in a_run_chat
    speaker = await groupchat.a_select_speaker(speaker, self)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nathan/miniconda3/envs/opspilot-query-auto-agent/lib/python3.11/site-packages/autogen/agentchat/groupchat.py", line 574, in a_select_speaker
    return await self.a_auto_select_speaker(last_speaker, selector, messages, agents)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nathan/miniconda3/envs/opspilot-query-auto-agent/lib/python3.11/site-packages/autogen/agentchat/groupchat.py", line 784, in a_auto_select_speaker
    result = await checking_agent.a_initiate_chat(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nathan/miniconda3/envs/opspilot-query-auto-agent/lib/python3.11/site-packages/autogen/agentchat/conversable_agent.py", line 1173, in a_initiate_chat
    await self.a_send(msg2send, recipient, request_reply=True, silent=silent)
  File "/home/nathan/miniconda3/envs/opspilot-query-auto-agent/lib/python3.11/site-packages/autogen/agentchat/conversable_agent.py", line 800, in a_send
    await recipient.a_receive(message, self, request_reply, silent)
  File "/home/nathan/miniconda3/envs/opspilot-query-auto-agent/lib/python3.11/site-packages/autogen/agentchat/conversable_agent.py", line 951, in a_receive
    reply = await self.a_generate_reply(sender=sender)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nathan/miniconda3/envs/opspilot-query-auto-agent/lib/python3.11/site-packages/autogen/agentchat/conversable_agent.py", line 2138, in a_generate_reply
    final, reply = await reply_func(
                   ^^^^^^^^^^^^^^^^^
  File "/home/nathan/miniconda3/envs/opspilot-query-auto-agent/lib/python3.11/site-packages/autogen/agentchat/conversable_agent.py", line 1496, in a_generate_oai_reply
    return await asyncio.get_event_loop().run_in_executor(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nathan/miniconda3/envs/opspilot-query-auto-agent/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nathan/miniconda3/envs/opspilot-query-auto-agent/lib/python3.11/site-packages/autogen/agentchat/conversable_agent.py", line 1494, in _generate_oai_reply
    return self.generate_oai_reply(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nathan/miniconda3/envs/opspilot-query-auto-agent/lib/python3.11/site-packages/autogen/agentchat/conversable_agent.py", line 1436, in generate_oai_reply
    extracted_response = self._generate_oai_reply_from_client(
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nathan/miniconda3/envs/opspilot-query-auto-agent/lib/python3.11/site-packages/autogen/agentchat/conversable_agent.py", line 1455, in _generate_oai_reply_from_client
    response = llm_client.create(
               ^^^^^^^^^^^^^^^^^^
  File "/home/nathan/miniconda3/envs/opspilot-query-auto-agent/lib/python3.11/site-packages/autogen/oai/client.py", line 777, in create
    response = client.create(params)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/home/nathan/miniconda3/envs/opspilot-query-auto-agent/lib/python3.11/site-packages/autogen/oai/client.py", line 342, in create
    response = completions.create(**params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nathan/miniconda3/envs/opspilot-query-auto-agent/lib/python3.11/site-packages/openai/_utils/_utils.py", line 275, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/nathan/miniconda3/envs/opspilot-query-auto-agent/lib/python3.11/site-packages/openai/resources/chat/completions.py", line 581, in create
    return self._post(
           ^^^^^^^^^^^
  File "/home/nathan/miniconda3/envs/opspilot-query-auto-agent/lib/python3.11/site-packages/openai/_base_client.py", line 1233, in post
    return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nathan/miniconda3/envs/opspilot-query-auto-agent/lib/python3.11/site-packages/openai/_base_client.py", line 922, in request
    return self._request(
           ^^^^^^^^^^^^^^
  File "/home/nathan/miniconda3/envs/opspilot-query-auto-agent/lib/python3.11/site-packages/openai/_base_client.py", line 1013, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'error': {'message': "Invalid parameter: messages with role 'tool' must be a response to a preceeding message with 'tool_calls'.", 'type': 'invalid_request_error', 'param': 'messages.[2].role', 'code': `None}}

This has started happening since i upgraded from pyautogen==0.2.26 to autogen-agentchat==0.2.36

Is there any solution to this as it is making my service very unstable?

What did you expect to happen?

Not to error out intermittently due to bad tool calls

How can we reproduce it (as minimally and precisely as possible)?

I am running a GroupChat instance with a single user proxy agent and a number of conversable agents using async tools and have auto set for speaker selection method.

It seems to happen more when running multiple threads executing the groupchat at once but that may just be coincidence

I am using gpt4o mini

AutoGen version

0.2.36

Which package was this bug in

AgentChat

Model used

gpt4o-mini

Python version

No response

Operating system

No response

Any additional info you think would be helpful for fixing this bug

No response

ekzhu commented 1 day ago

can you provide a script so we can reproduce this? Without knowing how you set up the group chat it is very hard for us to understand the source of the error.

From your description and stack trace. It seems that the order of messages in the group chat manager might have been scrambled. A tool call should be followed by a tool call result message.