[ ] Orchestrating Agents: Routines and Handoffs | OpenAI Cookbook

Orchestrating Agents: Routines and Handoffs | OpenAI Cookbook

Routines

A routine is defined as a list of instructions in natural language (represented by a system prompt), along with the tools necessary to complete them. Here's an example of a customer service routine:

# Customer Service Routine

system_message = (
    "You are a customer support agent for ACME Inc."
    "Always answer in a sentence or less."
    "Follow the following routine with the user:"
    "1. First, ask probing questions and understand the user's problem deeper.\n"
    " - unless the user has already provided a reason.\n"
    "2. Propose a fix (make one up).\n"
    "3. ONLY if not satesfied, offer a refund.\n"
    "4. If accepted, search for the ID and then execute refund."
    ""
)

def look_up_item(search_query):
    """Use to find item ID.
    Search query can be a description or keywords."""

    # return hard-coded item ID - in reality would be a lookup
    return "item_132612938"

def execute_refund(item_id, reason="not provided"):
    print("Summary:", item_id, reason) # lazy summary
    return "success"

The power of routines lies in their simplicity and robustness. LLMs can handle these conditional instructions quite robustly for small and medium-sized routines, with the added benefit of "soft" adherence - the LLM can naturally steer the conversation without getting stuck in dead-ends.

Handoffs

A handoff is defined as an agent (or routine) handing off an active conversation to another agent, much like when you get transferred to someone else on a phone call. The agents have complete knowledge of the prior conversation.

To implement handoffs, we define an Agent class and modify the run_full_turn function to handle agent transfers:

class Agent(BaseModel):
    name: str = "Agent"
    model: str = "gpt-4o-mini"
    instructions: str = "You are a helpful Agent"
    tools: list = []

def run_full_turn(agent, messages):
    # ...
    while True:
        # ...
        for tool_call in message.tool_calls:
            result = execute_tool_call(tool_call, tools_map, agent.name)
            if type(result) is Agent:  # if agent transfer, update current agent
                agent = result
                result = f"Transferred to {agent.name}. Adopt persona immediately."
            # ...
    return Response(agent=agent, messages=messages[num_init_messages:])

Agents can express the intent to make a handoff by returning an Agent object from their tool functions. The run_full_turn function then updates the current agent accordingly.

Example

Here's an example with multiple agents:

triage_agent = Agent(
    name="Triage Agent",
    instructions=(
        "You are a customer service bot for ACME Inc. "
        "Introduce yourself. Always be very brief. "
        "Gather information to direct the customer to the right department. "
        "But make your questions subtle and natural."
    ),
    tools=[transfer_to_sales_agent, transfer_to_issues_and_repairs, escalate_to_human],
)

sales_agent = Agent(
    name="Sales Agent",
    instructions=(
        "You are a sales agent for ACME Inc."
        "Always answer in a sentence or less."
        "Follow the following routine with the user:"
        "1. Ask them about any problems in their life related to catching roadrunners.\n"
        "2. Casually mention one of ACME's crazy made-up products can help.\n"
        " - Don't mention price.\n"
        "3. Once the user is bought in, drop a ridiculous price.\n"
        "4. Only after everything, and if the user says yes, "
        "tell them a crazy caveat and execute their order.\n"
        ""
    ),
    tools=[execute_order, transfer_back_to_triage],
)

issues_and_repairs_agent = Agent(
    name="Issues and Repairs Agent",
    instructions=(
        "You are a customer support agent for ACME Inc."
        "Always answer in a sentence or less."
        "Follow the following routine with the user:"
        "1. First, ask probing questions and understand the user's problem deeper.\n"
        " - unless the user has already provided a reason.\n"
        "2. Propose a fix (make one up).\n"
        "3. ONLY if not satesfied, offer a refund.\n"
        "4. If accepted, search for the ID and then execute refund."
        ""
    ),
    tools=[execute_refund, look_up_item, transfer_back_to_triage],
)

The agents can then be used in a loop, allowing the user to interact with the different agents as needed:

agent = triage_agent
messages = []

while True:
    user = input("User: ")
    messages.append({"role": "user", "content": user})

    response = run_full_turn(agent, messages)
    agent = response.agent
    messages.extend(response.messages)

Swarm

As a proof of concept, these ideas have been packaged into a sample library called Swarm. It is meant as an example only and should not be directly used in production. However, feel free to take the ideas and code to build your own.

irthomasthomas / undecidability