[Bug]: custom `speaker_selection_method` does not use `register_nested_chats`

[Morriz]

Describe the bug

Hi, I assume nested chats to work even when using a custom speaker_selection_method, as it is nested and should follow the configuration

Steps to reproduce

from agents.writing_agent import WritingAgent
from autogen import AssistantAgent, GroupChat, GroupChatManager, UserProxyAgent

agency = "book-agency"

config = {
    "config_list": [
        {
            "model": "gpt-4o",
        },
    ],
    "cache_seed": None,  # Disable caching.
    "temperature": 0,
    "timeout": 120,
}

user_proxy = UserProxyAgent(
    name="user",
    human_input_mode="NEVER",
    is_termination_msg=lambda x: x.get("content", "").find("TERMINATE") >= 0,
    code_execution_config={
        "last_n_messages": 1,
        "work_dir": f"{agency}/output",
        "use_docker": False,
    },
)

editor = AssistantAgent(
    name="editor",
    llm_config={**config, "temperature": 0.5},
    system_message="""
    You are a senior editor, and know all about the best books out there. You have extensive experience and worked with the best in the field.
    You are known for your thoroughness and commitment to producing interesting books of the highest quality.
    You will assist the team in your book agency.
    Your task is to give thorough feedback on the last agent's output. You are on a deadline, so stay calm but be exhaustive.
    Your team might win an award for this one, so make it count!
    """,
)

writer = AssistantAgent(
    name="writer",
    llm_config={**config, "temperature": 1},
    system_message=f"""You are a great writer with tons of experience. You know all the famous books, short stories and essays.
    You will find your asignment in the input. Your writings should have original takes on the input. You will think of the story arch before you start writing.
    You may use all kinds of humor, including satire, irony, sarcasm, slapstick, etc, but of course there should be some tone of seriousness to it all as we are writing a dramedy.
    """,
)

# Define a reflection message for the editor to provide feedback on the writer's output
def writer_reflection_message(recipient, messages, sender, config):
    print("Reflecting on story...", "yellow")
    return f"Reflect and provide critique on the following story. \n\n {recipient.chat_messages_for_summary(sender)[-1]['content']}"

# Register the editor as critic for the writer
user_proxy.register_nested_chats(
    [
        {
            "recipient": editor,
            "message": writer_reflection_message,
            "summary_method": "last_msg",
            "max_turns": 2,
        }
    ],
    trigger=writer,
)

def speaker_selection_method(last_speaker, groupchat):

    speaker = {
        writer: None,
    }

    return speaker.get(last_speaker, writer)

groupchat = GroupChat(
    agents=[
        user_proxy,
        editor,
        writer,
    ],
    messages=[],
    max_round=20,
    speaker_selection_method=speaker_selection_method,
)
manager = GroupChatManager(groupchat=groupchat, llm_config=config)

input_data = """
short story of max 5 pages:
- family with 24 children
- inherited a fortune plus many large real estates around the world
- live in a castle with lots of spare rooms even with all the kids having their own
- castle is on a plot of 300 acres with beautiful gardens and remote hideaways
"""

# Pass input to initializer
user_proxy.initiate_chat(
    manager,
    message=input_data,
)

Model Used

No response

Expected Behavior

nested chats to be called before anything else that is returned from state_transition

Screenshots and logs

No response

Additional Information

No response

[ekzhu]

Could you explain a bit more what do you mean by

nested chats to be called before anything else that is returned from state_transition

The state_transition function is only choosing the next speaker.

[Morriz]

Like I said, I would assume that the nested chat gets initiated, but the provision of the custom speaker_selection_method borks that

[ekzhu]

I am not getting your point. speaker_selection_method operates at the level of group chat, not individual agents.

In your code example, you need to change the trigger of the nested chat to the group chat manager to trigger the nested chat.