microsoft / autogen

A programming framework for agentic AI 🤖
https://microsoft.github.io/autogen/
Creative Commons Attribution 4.0 International
32.37k stars 4.72k forks source link

GroupChat is overly sensitive to formatting when selecting the next speaker. #663

Closed afourney closed 11 months ago

afourney commented 11 months ago

As pointed out by Jonny7377 on Discord, GroupChat is overly sensitive to formatting issues when selecting the next speaker. E.g.,

GroupChat select_speaker failed to resolve the next speaker's name. Speaker selection will default to the next speaker in the list. This is because the speaker selection OAI call returned:
 Coder

In this case, the extra space causes the lookup to fail.

At the very least, the code should strip() whitespace from the agent names before doing a lookup. That can be done here:

https://github.com/microsoft/autogen/blob/4711260582536e23ebbd9c3ccd067cddf60cd21f/autogen/agentchat/groupchat.py#L42

Or here,

https://github.com/microsoft/autogen/blob/4711260582536e23ebbd9c3ccd067cddf60cd21f/autogen/agentchat/groupchat.py#L90

Alternatively, a more robust option might be to check if exactly one agent is mentioned in response, and if so, select it (regardless of how it is formatted).

AndreiD commented 10 months ago

still have the same issue

asandez1 commented 6 months ago

The issue arises when utilizing an open-source language model (LLM), whereas switching to OpenAI resolves the error. "GroupChat select_speaker failed to resolve the next speaker's name"

Change the LLM to the GroupChatManager ,and the agents can continue using the open-source llm

sonichi commented 6 months ago

2304 is an effort to improve this.

marklysze commented 6 months ago

I'm working on #2304 and have included strip() on the return result. I've updated what is in #2304 to use agents for the speaker selection and am testing before checking that back in. This issue will be covered by that work.