microsoft / autogen

A programming framework for agentic AI 🤖
https://microsoft.github.io/autogen/
Creative Commons Attribution 4.0 International
34.66k stars 5.01k forks source link

Make `Swarm` work with `HandoffTermination` #4199

Closed ekzhu closed 4 days ago

ekzhu commented 1 week ago

Currently Swarm does not work with HandoffTermination, because the handoff message to user or a name that is not part of the swarm's participants will trigger an error when choosing the next speaker.

The caller must provide a new HandoffMessage in response when resuming the same thread, to make sure the handoff to the next target is properly done. This requires:

  1. run and run_stream accepting ChatMessage type.
  2. validation and informative error message when a HandoffMessage is not provided.
gziz commented 1 week ago

Thought of something a bit hacky but working, might give some inspiration...

Custom UserProxyAgent based on BaseChatAgent, that returns handoff and user reply as messages.

Basically, let on_messages return the user reply as Reponse and the handoff message as inner_message

Why are we sending the handoff_message as inner, because, if we send the user reply as inner, this text won’t be present in the messages history for the other agents, it will be as if the user never replied. However, handoff_message not showing here is no problem since we only need it for the select_speaker. We would include the 'user' as an available handoff for the other agents.

class UserProxyAgent(BaseChatAgent):
    def __init__(self, name: str) -> None:
        super().__init__(name, "A human user.")

    @property
    def produced_message_types(self) -> List[type[ChatMessage]]:
        return [TextMessage, StopMessage, HandoffMessage]

    async def on_messages(
        self, messages: Sequence[ChatMessage], cancellation_token: CancellationToken
    ) -> Response:
        user_input = await asyncio.get_event_loop().run_in_executor( None, input, "User: ")

        # Find the last HandOffMessage targeting the user to determine who to hand off to.
        last_handoff_source = None
        for message in reversed(messages):
            if isinstance(message, HandoffMessage) and message.target == self.name:
                last_handoff_source = message.source
                break

        text_message = TextMessage(content=user_input, source=self.name)
        handoff_message = HandoffMessage(
            content="Handoff back to agent",
            source=self.name,
            target=last_handoff_source,
        )
        return Response(chat_message=text_message, inner_messages=[handoff_message])

    async def on_reset(self, cancellation_token: CancellationToken) -> None:
        self._last_handoff_source = None
ekzhu commented 6 days ago

Custom UserProxyAgent based on BaseChatAgent, that returns handoff and user reply as messages.

I believe we need this for the documentation of Swarm.