Open milioe opened 8 months ago
It looks like there are two problems here, and it is the Notebook that needs to be fixed.
The first problem is that ExecutorGroupchat is subclassing GroupChat, but GroupChat has evolved since this was written. Many of the methods require an agents parameter now, because the list can be dynamic due to new allow_repeat_speaker
options. I can fix this by setting it as an optional parameter in GroupChat, then setting sane defaults. But, we probably also want the original author of this notebook to fix things. I'll track down who that was.
The second problem is that you are passing an llm_config with functions to the GroupChatManager -- that's not supported behavior (and was the source of several breaking bugs). This is why you are now seeing the exception. Create a new llm_config that you pass to the GroupChatManager. Just set it to:
manager_llm_config = {
"config_list": config_list,
"seed": 45,
"request_timeout": 120
}
@sonichi @kevin666aa for visibility
@afourney - When I use a different config for group chat manager that does not include the function definition. But when the function executor responds with a response from the function that is run, the chat manager errors out that the role = 'function' is not supported
Encountered BadRequestError('Error code: 400 - {\'error\': {\'code\': \'BadRequest\', \'message\': "\'function\' is not an allowed role. The allowed roles are [\'system\', \'user\', \'assistant\'].", \'param\': None, \'type\': None}}'). Set callback_exception='verbose' to see the full traceback.
I am not sure where I can set the callback_exception='verbose' to see additional logging here as well.
@kbalasu1 I suggest opening a separate case for this - but here is my (uneducated) pointer:
functions
definitions is passed in.I may not see the whole picture here - but anyway @afourney maybe this could be helpful.
@kbalasu1 would be useful if you provide a minimal reproduction code (including explicit llm_config if possible and without the api_key of course) so we can run locally and trace the problem.
Btw, code here and here also needs to generalize to "tool" and not just "function" - Maybe worth having a "is_tool_role" helper function that can be used and internally handle the specific two possible roles to reduce bugs like theses.
@yoadsn Thanks for the notes. I agree with most of them. Your first point is valid, and the last suggestion is valid too.
https://github.com/microsoft/autogen/blob/2e519b016a8bfa7a807721a1fc8a93b5d3be6c32/autogen/agentchat/conversable_agent.py#L702 is needed for tool responses only because we store tool responses in a special way that needs to be processed.
To address the first point, which is the hardest, I'm thinking of the following: For agents who are neither the suggester nor the executor of a function/tool, the function/tool call/response message needs to be processed into plain text before sending to them by the group chat manager. Same for the groupchat
's message which will be used to perform speaker selection.
After investigation, the described bug doesn't exist for sync group chat. There is indeed a bug for async group chat due to mismatch of the sync group chat. #1243 fixes that and add tests. The last comment about generalization to "tool" is also not necessary as seen in these tests.
I'm having the similar issue and my code is very similar to the @milioe , I checked the code @yoadsn shared from the groupchat.py, should "function_map" be used instead of "functions"? if so can you point me to the right place please @sonichi ? thank you
if I am not mistaken, in an earlier version of autogen, didn't we have to call the "pop" on the llm_config right before assigning it to the groupchat? that way we pop out functions and tools? I remember having to do that but don't remember the version. Please can someone check.
Thanks.
Just a quick update, I found the fix. Not sure how you all will implement it but I went to the Autogen github website and found an example and I added it and it worked for me as I had this same issue with autogen version 0.2.20 . I came across this issue working on the sample notebook agentchat_groupchat_RAG.ipynb .
https://microsoft.github.io/autogen/docs/notebooks/agentchat_function_call_async
The code is below and notice how I used the pop function and renamed the groupchat manager llm_config so there is nothing confusing to me. Again, I remembered this being an issue from 0.1.8 version. Please someone fix this for the next iteration if you can. It's definitely a headache to figure out for such a simple solution.
`llm_config = { "timeout": 600, "temperature": 0, "config_list": config_list, }
def termination_msg(x): return isinstance(x, dict) and "TERMINATE" == str(x.get("content", ""))[-9:].upper()
boss = autogen.UserProxyAgent(
name="Boss",
is_termination_msg=termination_msg,
human_input_mode="NEVER",
code_execution_config=False, # we don't want to execute code in this case.
default_auto_reply="Reply TERMINATE
if the task is done.",
description="The boss who ask questions and give tasks.",
)
boss_aid = RetrieveUserProxyAgent( name="Boss_Assistant", is_termination_msg=termination_msg, human_input_mode="NEVER", max_consecutive_auto_reply=3, retrieve_config={ "task": "code", "docs_path": "https://raw.githubusercontent.com/microsoft/FLAML/main/website/docs/Examples/Integrate%20-%20Spark.md", "chunk_token_size": 1000, "model": config_list[0]["model"], "client": chromadb.PersistentClient(path="/tmp/chromadb"), "collection_name": "flaml", "get_or_create": True, }, code_execution_config=False, # we don't want to execute code in this case. description="Assistant who has extra content retrieval power for solving difficult problems.", )
coder = AssistantAgent(
name="Senior_Python_Engineer",
is_termination_msg=termination_msg,
system_message="You are a senior python engineer, you provide python code to answer questions. Reply TERMINATE
in the end when everything is done.",
llm_config=llm_config,
description="Senior Python Engineer who can write code to solve problems and answer questions.",
)
pm = autogen.AssistantAgent(
name="Product_Manager",
is_termination_msg=termination_msg,
system_message="You are a product manager. Reply TERMINATE
in the end when everything is done.",
llm_config=llm_config,
description="Product Manager who can design and plan the project.",
)
reviewer = autogen.AssistantAgent(
name="Code_Reviewer",
is_termination_msg=termination_msg,
system_message="You are a code reviewer. Reply TERMINATE
in the end when everything is done.",
llm_config=llm_config,
description="Code Reviewer who can review the code.",
)
PROBLEM = "How to use spark for parallel training in FLAML? Give me sample code."
def _reset_agents(): boss.reset() boss_aid.reset() coder.reset() pm.reset() reviewer.reset()
def rag_chat(): _reset_agents() groupchat = autogen.GroupChat( agents=[boss_aid, pm, coder, reviewer], messages=[], max_round=12, speaker_selection_method="round_robin" ) manager = autogen.GroupChatManager(groupchat=groupchat, llm_config=llm_config)
# Start chatting with boss_aid as this is the user proxy agent.
boss_aid.initiate_chat(
manager,
message=boss_aid.message_generator,
problem=PROBLEM,
n_results=3,
)
def norag_chat(): _reset_agents() groupchat = autogen.GroupChat( agents=[boss, pm, coder, reviewer], messages=[], max_round=12, speaker_selection_method="auto", allow_repeat_speaker=False, ) manager = autogen.GroupChatManager(groupchat=groupchat, llm_config=llm_config)
# Start chatting with the boss as this is the user proxy agent.
boss.initiate_chat(
manager,
message=PROBLEM,
)
def call_rag_chat(): _reset_agents()
# In this case, we will have multiple user proxy agents and we don't initiate the chat
# with RAG user proxy agent.
# In order to use RAG user proxy agent, we need to wrap RAG agents in a function and call
# it from other agents.
def retrieve_content(
message: Annotated[
str,
"Refined message which keeps the original meaning and can be used to retrieve content for code generation and question answering.",
],
n_results: Annotated[int, "number of results"] = 3,
) -> str:
boss_aid.n_results = n_results # Set the number of results to be retrieved.
# Check if we need to update the context.
update_context_case1, update_context_case2 = boss_aid._check_update_context(message)
if (update_context_case1 or update_context_case2) and boss_aid.update_context:
boss_aid.problem = message if not hasattr(boss_aid, "problem") else boss_aid.problem
_, ret_msg = boss_aid._generate_retrieve_user_reply(message)
else:
_context = {"problem": message, "n_results": n_results}
ret_msg = boss_aid.message_generator(boss_aid, None, _context)
return ret_msg if ret_msg else message
boss_aid.human_input_mode = "NEVER" # Disable human input for boss_aid since it only retrieves content.
for caller in [pm, coder, reviewer]:
d_retrieve_content = caller.register_for_llm(
description="retrieve content for code generation and question answering.", api_style="function"
)(retrieve_content)
for executor in [boss, pm]:
executor.register_for_execution()(d_retrieve_content)
groupchat = autogen.GroupChat(
agents=[boss, pm, coder, reviewer],
messages=[],
max_round=12,
speaker_selection_method="round_robin",
allow_repeat_speaker=False,
)
llm_config_manager = llm_config.copy()
llm_config_manager.pop("functions", None)
llm_config_manager.pop("tools", None)
manager = autogen.GroupChatManager(groupchat=groupchat, llm_config=llm_config_manager)
# Start chatting with the boss as this is the user proxy agent.
boss.initiate_chat(
manager,
message=PROBLEM,
)`
Don't share llm_config among different agents. You can share config list but LLM configs gets modified as you register functions
@ekzhu and @afourney Thanks for explaining that. I was also making it known that this is reproducible directly from Microsoft Autogen's sample notebook under the current version, of which I came across this issue for the agentchat_groupchat_RAG.ipynb . Thankfully, I remembered what was being done from the old autogen==0.1.8 version to remedy the issue, but your answers are the correct answer and they explain thoroughly why this error is happening, and it is unfortunate that based on the sample notebooks, this is unknown.
Can this be changed in the documentation and sample notebooks?
Thanks in advance
Yup. Can you file a documentation bug. We're going through and updating all the documentation now, and I don't want to lose track of this.
-- Adam
Yes, I will file a documentation bug for this today.
Describe the bug
Hi,
with pyautogen==0.2.2 I tried to run three codes provided in #274 and previously in #252 and #152 about function calling in groupchats but I got errors from all of them.
Steps to reproduce
This is my current code:
i got this error on three options:
Expected Behavior
I noticed that if i pass the
llm_config
argument tomanager
(manager = GroupChatManager(groupchat=groupchat, llm_config=llm_config, system_message="Choose one agent to play the role of the user proxy"
) this error appears:ValueError: GroupChatManager is not allowed to make function/tool calls. Please remove the 'functions' or 'tools' config in 'llm_config' you passed in.
Screenshots and logs
No response
Additional Information
I understand that when #152 was raised as an issue autogen was > 0.1.8 and #274 was > v0.1.11.
Is it because the versions, the code or any of the prompts? I'd appreciate your help.