Works as Autogen script, but fails in AutogenStudio Worklow - Group - As an AI language model, I don't have the capability to directly access external databases or URL[Bug]:

Describe the bug

Hi, I am trying to setup a 3 agent team (2 external and 1 user agent) with a simple case to access data from a URL. When running the script from either a .py file or as a jupyter notebook -- there is no problem.

The issue occurs only when trying to set this up as a Workflow --> Group when it consistently fails with t the. message "I apologize for the confusion earlier. As an AI language model, I don't have the capability to directly access external databases or URLs to retrieve or interact with data in real-time. However, I can guide you on how to extract the data using a programming language like Python. Screenshot 2024-02-29 at 10 46 58 AM "

Steps to reproduce

Screenshot 2024-02-29 at 10 47 52 AM

Expected Behavior

the workflow fails without extracting data from the URL -- this is likely to be a bug as the feature works fine when running this same group as a python script or jupyter notebook

Screenshots and logs

Screenshot 2024-02-29 at 10 46 58 AM Screenshot 2024-02-29 at 10 48 03 AM Screenshot 2024-02-29 at 10 47 52 AM

Additional Information

No response

Hi @gaurav8936 ,

There is a related issue here #1800 and #1423 .

For groupchat to execute code, you will need to add a user proxy to the group chat that can receive and execute code, and also add the skill to agent.

====

see #1423

The way GroupChat is configured is critical to the behaviors you see. Understanding AutoGen agent class behaviors is valuable here. I wrote an article here. The relevant section is:

UserProxyAgent as code executor: Use the userProxyAgent as a code executor (it should have a code_execution_config) that serves to execute code in any message it receives and then shares the result with other agents.
AssistantAgent as the main work horse: The AssistantAgent should be primed to implement the core behaviors you want to achieve. E.g. if you want to generate charts, retrieve documents, call functions etc, these should be specified in the system message of an AssistantAgent. Also specific skills (either as function calls or code added to system message) should be added to the assistant agent (not the user proxy). In writing prompts for your assistantFrom this perspective, the AssistantAgent should not have code_execution_config.
Extend the Default Assistant Prompt: In writing prompts for your assistant, a good first step is to extend the the default assistant prompt and add your own instructions. The main benefit here is that the the default assistant prompt instructs the agent to create a plan and write code to solve the task. This is important as the this code writte is received by the user proxy and then executed. If no code is written, the user proxy does not get any code to execute an nothing happens. (@saibhaskerraju)

As an example, I am attaching a sample workflow config that shows a team structured to generate art content. 3 agents

art director - assistant agent that plans the art pieces to be created, generates ideas
art implementer -- assistant agent that writes the code that implements the ideas
user proxy -- executes code (your group chat should have a user proxy)

Results are in the attached video.

workflow_Painter Team.json

You can upload this file in autogenstudio

https://github.com/microsoft/autogen/assets/1547007/79fc2876-7ccf-4e5f-87ac-365b0bc85df0

Let me know if this is helpful.

Hey @victordibia , I'm not the OP but I came here with a very similar issue. Your message and article set me straight in a few ways in regards to userproxy. It was not clear at all that this is the user that needs to be executing code. To note, I am using autogen studio. It is presented that you write code as a skill, then assign it to any agent. So if I wanted to pull text from a website, I would write that code and give it to 'text_puller' agent. But it sounds like from your post that we need to run all of that through the user_proxy.

To summarize my confusion:

1) Between the primary_assistant and the user_proxy, it sounds like these should be loaded with any and all skills that are relevant to the workflow. Is this right? It sounds like primary_assistant can write or suggest code, but then user_proxy is the one who executes it? Therefore they both need to know about the skills available? You mention the code_execution_config in your post, but that is not something available in the GUI. user_proxy looks just like any other user and other than the description stating that it can run code, there is no "code_execution" skill or such assigned to it. As a studio user, that is something I found very confusing, as it's not called out in the UI.

2) Not only did user_proxy need to be the 'sender' in my group chats, I had to add him in to the receiver list in order for him to actually attempt anything. I think perhaps my issue here was most videos I saw only walk through a two agent workflow, and not a group chat. I think that was something very important that is not clear.

@victordibia I concurr with @POD319 I'm seeing very weird behaviours and this seems overly complicated, should'nt be each agent be able to run code on their own? Having to force it to the user proxy kinda defeats the purpose. I have a similar case that for some reason user_proxy is never called and no code is executed. Please we need guidance.

@victordibia and @POD319 just tried again and it seems that after adding a skill to an agent you need to remove userproxy and add it again in the interface, with adjustments to my group chat manager, telling him that all code should be managed by userproxy agent and removing/adding again user proxy agent it seems to be working. There are some weird behaviours and we need to tell explicitly to each agent that they need to show the code to be executed. So Agent A has the skill, it outputs the skill code and then it userproxy takes the relay and executes the code. Doesn't seem intuitive.

FreddyAyala

Thanks @FreddyAyala - I think I ended up having the same result. There are some major UI issues that keep coming up, so I am looking forward to newer releases.

I agree that it is never made clear in the UI that only userproxy can execute code. I don't understand it either -- we should be able to give any agent a skill, and if that means executing code, the agent should be able to do it if we allow.

microsoft / autogen