search

microsoft / autogen

A programming framework for agentic AI 🤖
https://microsoft.github.io/autogen/
Creative Commons Attribution 4.0 International
34.35k stars 4.96k forks source link

How to set multiple agents flow in the Groupchat #584

Closed NikhilModi488 closed 7 months ago

NikhilModi488 commented 1 year ago

Hello @iemejia @lgersman @qinc @msaelices @craigomatic ,

I want to establish the flow for all the agents in the group chat. How can I configure the flow?

I have 8 agents and I want to set the flow of all the 8 agents. some agents work based on the other agent. How can i set the flow of the agents?

afourney commented 1 year ago

I believe this is related to:

485

489

565

And PR:

500

joshkyh commented 1 year ago

In this approach, I overridden the select_speaker method of the GroupChat class.

https://github.com/microsoft/autogen/blob/main/notebook/agentchat_hierarchy_flow_using_select_speaker.ipynb

afourney commented 1 year ago

Yes, I suggested this notebook to the PR author (#500)

Morriz commented 10 months ago

Any native construct in the work as opposed to asking users to override classes?

afourney commented 10 months ago

Now you can set the selection_mode to round robin and then it just goes in the order that you specify in the list. There is also this PR that addresses the problem #857

Morriz commented 10 months ago

So it is currently not possible to model real world workflows? Naive example:

  1. PM: create product x
  2. Dev: created x
  3. QA: critique means rework
  4. Dev: reworked x
  5. QA: ok now
  6. PM: demonstrate product x
  7. Dev: demoing x
  8. PM: signoff

This could be realized by defining relationships:

  1. at a higher level
  2. in the agent context, allowing the actor to say who's next (if critique call Dev, else PM)

First thought: nr 1 is preferred as 2 implies using that flaky speaker selection mechanism, and it means modeling in context, which might be unnecessary and all over the place.

Second thought: I am actually thinking now that modeling relationships into each agent's context has the added advantage of bringing in the meaning of the relationships, which might produce better results, because the actor is aware of the others and their role.

Final thought: In the end I think the best results come from in context relationship modeling for better inference, but with a higher level relationship configuration for the actual speaker selection.

sonichi commented 10 months ago

@joshkyh fyi

j4ys0n commented 7 months ago

So it is currently not possible to model real world workflows? Naive example:

  1. PM: create product x
  2. Dev: created x
  3. QA: critique means rework
  4. Dev: reworked x
  5. QA: ok now
  6. PM: demonstrate product x
  7. Dev: demoing x
  8. PM: signoff

this is basically what i'm trying to do also and i keep getting GroupChat select_speaker failed to resolve the next speaker's name. This is because the speaker selection OAI call returned: and it ends up doing something like a round robin but sends each agent 2 tokens from the response to the original prompt that's streaming in instead of gathering the response from agent 1 and sending it to group for the groupchat_assistant to figure out the next agent to send it to.

afourney commented 7 months ago

There are a few issues possibly at play here. Since the original issue, graph group chat, initiate chats, and nested chats should each support this type of orchestration. The group chat's failure to name an agents feels like a different problem. What model are you using @j4ys0n , and how are the agents configured?

j4ys0n commented 7 months ago

thanks for the response @afourney.

yeah - it could be. been trying to narrow it down for a while.

i'm using the UI so i've gotta grab the config from the dev inspector, but i believe this has the full config:

{
    "status": true,
    "message": "Agents retrieved successfully",
    "data": [
        {
            "type": "assistant",
            "config": {
                "name": "optimizer",
                "llm_config": {
                    "config_list": [
                        {
                            "id": "1288cdd6-42c2-44fb-8127-96b2f4b5c693",
                            "user_id": "default",
                            "timestamp": "2024-03-31T20:01:13.657199",
                            "model": "dolphin 2.5 mixtral 8x 7b q4km",
                            "api_key": "key",
                            "base_url": "http://localhost:1234/v1",
                            "api_type": "openai",
                            "api_version": null,
                            "description": ""
                        }
                    ],
                    "temperature": 0.1,
                    "cache_seed": null,
                    "timeout": null,
                    "max_tokens": null,
                    "extra_body": null
                },
                "human_input_mode": "NEVER",
                "max_consecutive_auto_reply": 30,
                "system_message": "As the Optimizer, your role is to refine and enhance the code produced by the Engineer. Your focus is on optimizing for speed, memory efficiency, and readability, without compromising the functionality or integrity of the code. Review the implementation for opportunities to abstract redundant functions, simplify complex logic, and eliminate any inefficiencies. Ensure that your optimizations maintain or improve the performance and maintainability of the code. Your goal is to produce the most efficient version of the solution that still clearly conveys its logic and structure for future maintenance. Keep the code concise and consider abstracting similar functions into more generalized implementations where applicable. After optimizing, pass your output back to the Reviewer for a final check, ending with \"TERMINATE\" to indicate completion.",
                "is_termination_msg": null,
                "code_execution_config": false,
                "default_auto_reply": "",
                "description": "the optimizer checks output from the engineer to ensure it's clear and efficient while still being correct and functional."
            },
            "id": "746832d0-c3e4-4c99-9e44-f49044a76777",
            "timestamp": "2024-04-02T00:29:04.996420",
            "user_id": "guestuser@gmail.com",
            "skills": [skills here...]
        },
        {
            "type": "assistant",
            "config": {
                "name": "reviewer",
                "llm_config": {
                    "config_list": [
                        {
                            "id": "1288cdd6-42c2-44fb-8127-96b2f4b5c693",
                            "user_id": "default",
                            "timestamp": "2024-03-31T20:01:13.657199",
                            "model": "dolphin 2.5 mixtral 8x 7b q4km",
                            "api_key": "key",
                            "base_url": "http://localhost:1234/v1",
                            "api_type": "openai",
                            "api_version": null,
                            "description": ""
                        }
                    ],
                    "temperature": 0.1,
                    "cache_seed": null,
                    "timeout": null,
                    "max_tokens": null,
                    "extra_body": null
                },
                "human_input_mode": "NEVER",
                "max_consecutive_auto_reply": 30,
                "system_message": "As the Reviewer, your responsibility is to meticulously evaluate the Architect's plan or the Engineer's code. Ensure the architectural designs are sensible, feasible, and optimized for the specified objectives. For plans, verify that they include clear, actionable steps for implementation in the correct languages and frameworks, integration points are well-defined, and complex functions are explained in detail. For code, check for completeness, correctness, syntax accuracy, and performance efficiency. If you find errors or areas for improvement, provide constructive feedback and suggestions. Your goal is to ensure zero bugs, optimal performance, and adherence to best practices. If the output is not up to standard, send it back with a detailed explanation of the issues and potential solutions. If the output meets all requirements, forward it to the next stage and indicate to the group which agent should pick up the task..",
                "is_termination_msg": null,
                "code_execution_config": false,
                "default_auto_reply": "",
                "description": "the reviewer double checks output from other agents to ensure they make sense and address the task."
            },
            "id": "04eaa59a-a573-4b83-85b6-b4150b82708d",
            "timestamp": "2024-04-02T00:28:39.025063",
            "user_id": "guestuser@gmail.com",
            "skills": [skills here...]
        },
        {
            "type": "assistant",
            "config": {
                "name": "architect",
                "llm_config": {
                    "config_list": [
                        {
                            "id": "1288cdd6-42c2-44fb-8127-96b2f4b5c693",
                            "user_id": "default",
                            "timestamp": "2024-03-31T20:01:13.657199",
                            "model": "dolphin 2.5 mixtral 8x 7b q4km",
                            "api_key": "key",
                            "base_url": "http://localhost:1234/v1",
                            "api_type": "openai",
                            "api_version": null,
                            "description": ""
                        }
                    ],
                    "temperature": 0.1,
                    "cache_seed": null,
                    "timeout": null,
                    "max_tokens": null,
                    "extra_body": null
                },
                "human_input_mode": "NEVER",
                "max_consecutive_auto_reply": 30,
                "system_message": "As the Architect, your role is to design comprehensive solutions that span frontend, backend, databases, caches, and cloud platforms, leveraging your extensive knowledge of libraries, packages, languages, and system integrations. You are tasked with conceptualizing the structure of software systems to ensure they are robust, scalable, and efficient. Outline the architecture of the solution, specifying the technologies and frameworks to be used, how different systems will interact, and any complex functions that need special attention. Provide detailed instructions for implementation, focusing on high-quality, maintainable, and performance-oriented code. Ensure your output is complete and precise, ready for the Reviewer to evaluate without needing further clarification. If you identify a need for information or functionality outside your immediate scope, consult the available Python functions and include calls to these functions where necessary. Your output will guide the Engineer in the implementation phase. Conclude your instructions with \"TERMINATE\" and indicate to the Reviewer that your instructions are ready for review.",
                "is_termination_msg": null,
                "code_execution_config": false,
                "default_auto_reply": "",
                "description": "the architect designs solutions that the engineer will implement."
            },
            "id": "d7323fc4-928d-468d-95bf-6006ad5fa1a0",
            "timestamp": "2024-04-02T00:27:52.148473",
            "user_id": "guestuser@gmail.com",
            "skills": [skills here...]
        },
        {
            "type": "assistant",
            "config": {
                "name": "engineer",
                "llm_config": {
                    "config_list": [
                        {
                            "id": "1288cdd6-42c2-44fb-8127-96b2f4b5c693",
                            "user_id": "default",
                            "timestamp": "2024-03-31T20:01:13.657199",
                            "model": "dolphin 2.5 mixtral 8x 7b q4km",
                            "api_key": "key",
                            "base_url": "http://localhost:1234/v1",
                            "api_type": "openai",
                            "api_version": null,
                            "description": ""
                        }
                    ],
                    "temperature": 0.1,
                    "cache_seed": null,
                    "timeout": null,
                    "max_tokens": null,
                    "extra_body": null
                },
                "human_input_mode": "NEVER",
                "max_consecutive_auto_reply": 30,
                "system_message": "As the Engineer, you are tasked with implementing the Architect's design to the highest standard. You possess comprehensive programming skills and are familiar with all relevant languages, frameworks, and technologies. Your focus is on writing clean, efficient, and well-documented code that is easy to read and maintain. Follow the Architect's instructions closely, ensuring all aspects of the design are translated into functioning code. Keep your code DRY and type all variables to prevent type-related errors. If you encounter challenges or uncertainties, consult the Architect's plan or the available Python functions for guidance. Once you have completed the implementation, review your work to ensure it meets the expected standards for performance, readability, and correctness. Conclude with \"TERMINATE\" and signal to the Reviewer that your output is ready for the Reviewer's inspection.",
                "is_termination_msg": null,
                "code_execution_config": false,
                "default_auto_reply": "",
                "description": "the engineer implements solutions that are designed by the architect."
            },
            "id": "71444a89-3fc4-46fb-9fd3-e7990b99d09f",
            "timestamp": "2024-04-01T01:45:26.289154",
            "user_id": "guestuser@gmail.com",
            "skills": [skills here...]
        },
        {
            "type": "userproxy",
            "config": {
                "name": "userproxy",
                "llm_config": false,
                "human_input_mode": "NEVER",
                "max_consecutive_auto_reply": 5,
                "system_message": "You are a helpful assistant.",
                "is_termination_msg": null,
                "code_execution_config": {
                    "work_dir": null,
                    "use_docker": false
                },
                "default_auto_reply": "TERMINATE",
                "description": "A user proxy agent that executes code."
            },
            "id": "d47d305b-4cd7-454d-a139-3fe8c5b81425",
            "timestamp": "2024-03-31T20:01:13.657265",
            "user_id": "default",
            "skills": null
        }
    ]
}
j4ys0n commented 7 months ago

it's very strange - no matter what i do, this is the result

userproxy (to groupchat_assistant):

i need a small, simple website for my consulting agency. the output should be a static website including javascript, html and css. we will focus on the exact content of it later, but for now i need a home page, contact page, about page and a products page. the products page should integrate with shopify's buy now button so that we can leverage and easy and familiar store experience for users. the contact page does not have to work at this time, a placeholder is fine for a contact form.

--------------------------------------------------------------------------------
No default IOStream has been set, defaulting to IOConsole.
Active Connections:  ['8f718b26-726d-494f-b67d-39ebe2f1e745']
Current message connection id:  8f718b26-726d-494f-b67d-39ebe2f1e745
GroupChat select_speaker failed to resolve the next speaker's name. This is because the speaker selection OAI call returned:

No default IOStream has been set, defaulting to IOConsole.
No default IOStream has been set, defaulting to IOConsole.
Request reply False sender_type groupchat silent False sender architect
No default IOStream has been set, defaulting to IOConsole.
architect (to groupchat_assistant):

 To create

--------------------------------------------------------------------------------
No default IOStream has been set, defaulting to IOConsole.
Active Connections:  ['8f718b26-726d-494f-b67d-39ebe2f1e745']
Current message connection id:  8f718b26-726d-494f-b67d-39ebe2f1e745
GroupChat select_speaker failed to resolve the next speaker's name. This is because the speaker selection OAI call returned:
 a small
No default IOStream has been set, defaulting to IOConsole.
No default IOStream has been set, defaulting to IOConsole.
Request reply False sender_type groupchat silent False sender engineer
No default IOStream has been set, defaulting to IOConsole.
engineer (to groupchat_assistant):

 a simple

--------------------------------------------------------------------------------
Active Connections:  ['8f718b26-726d-494f-b67d-39ebe2f1e745']
Current message connection id:  8f718b26-726d-494f-b67d-39ebe2f1e745
No default IOStream has been set, defaulting to IOConsole.
GroupChat select_speaker failed to resolve the next speaker's name. This is because the speaker selection OAI call returned:
 website,
No default IOStream has been set, defaulting to IOConsole.
No default IOStream has been set, defaulting to IOConsole.
Request reply False sender_type groupchat silent False sender optimizer
No default IOStream has been set, defaulting to IOConsole.
optimizer (to groupchat_assistant):

Active Connections:  ['8f718b26-726d-494f-b67d-39ebe2f1e745']
Current message connection id:  8f718b26-726d-494f-b67d-39ebe2f1e745
 website for

--------------------------------------------------------------------------------
No default IOStream has been set, defaulting to IOConsole.
GroupChat select_speaker failed to resolve the next speaker's name. This is because the speaker selection OAI call returned:
 your consulting
No default IOStream has been set, defaulting to IOConsole.
No default IOStream has been set, defaulting to IOConsole.
Request reply False sender_type groupchat silent False sender reviewer
No default IOStream has been set, defaulting to IOConsole.
reviewer (to groupchat_assistant):

 your consulting

--------------------------------------------------------------------------------
Active Connections:  ['8f718b26-726d-494f-b67d-39ebe2f1e745']
Current message connection id:  8f718b26-726d-494f-b67d-39ebe2f1e745
No default IOStream has been set, defaulting to IOConsole.
GroupChat select_speaker failed to resolve the next speaker's name. This is because the speaker selection OAI call returned:
 agency,
No default IOStream has been set, defaulting to IOConsole.
No default IOStream has been set, defaulting to IOConsole.
Request reply False sender_type groupchat silent False sender architect
No default IOStream has been set, defaulting to IOConsole.
architect (to groupchat_assistant):

 agency,

--------------------------------------------------------------------------------
ekzhu commented 7 months ago

@j4ys0n can you create a separate issue for your question?

The original issue regarding to agent flow should be resolved with the introduction of constrained speaker selection state flow:

  1. https://microsoft.github.io/autogen/docs/tutorial/conversation-patterns#constrained-speaker-selection
  2. https://microsoft.github.io/autogen/docs/topics/groupchat/customized_speaker_selection