microsoft / autogen

A programming framework for agentic AI. Discord: https://aka.ms/autogen-dc. Roadmap: https://aka.ms/autogen-roadmap
https://microsoft.github.io/autogen/
Creative Commons Attribution 4.0 International
29.89k stars 4.35k forks source link

[bug]the sanitize_agent_spec is called multiple times, the system_prompts has duplicate content. #1898

Open cillyfly opened 5 months ago

cillyfly commented 5 months ago

https://github.com/microsoft/autogen/blob/2e51cb3f6a23e75b50736762f4ac386e62a24f16/samples/apps/autogen-studio/autogenstudio/workflowmanager.py#L194

when it is groupchat will be call twice sanitize_agent_spec

        agent_spec = self.sanitize_agent_spec(agent_spec)
        if agent_spec.type == "groupchat":
            agents = [
                self.load(self.sanitize_agent_spec(agent_config)) for agent_config in agent_spec.groupchat_config.agents
            ]

agent_spec.config.system_message is a global parameter if this Function called twice system_message will has duplicate content.

https://github.com/microsoft/autogen/blob/2e51cb3f6a23e75b50736762f4ac386e62a24f16/samples/apps/autogen-studio/autogenstudio/workflowmanager.py#L175

        if agent_spec.skills:
            # get skill prompt, also write skills to a file named skills.py
            skills_prompt = ""
            skills_prompt = get_skills_from_prompt(agent_spec.skills, self.work_dir)
            if agent_spec.config.system_message:
                agent_spec.config.system_message = agent_spec.config.system_message + "\n\n" + skills_prompt
            else:
                agent_spec.config.system_message = get_default_system_message(agent_spec.type) + "\n\n" + skills_prompt
cillyfly commented 5 months ago

change adding skill_prompts to updating skill_prompts can solve this bug, but using magic text "solving the task you may use functions below which will be available in a file called skills.py " as a flag,It doesn't seem like a good solution

example

victordibia commented 5 months ago

Hi @cillyfly ,

Thanks for noting this.
I'll investigate this more tomorrow.

victordibia commented 5 months ago

Also ..

when it is groupchat will be call twice sanitize_agent_spec

Can you clarify what you mean here?

Do you you mean the groupchat manager ends up with duplicate content?

cillyfly commented 5 months ago

Also ..

when it is groupchat will be call twice sanitize_agent_spec

Can you clarify what you mean here?

  • agent_spec = self.sanitize_agent_spec(agent_spec) .. this is for GroupChatManager spec
  • agents = [ self.load(self.sanitize_agent_spec(agent_config)) ... this is for agents in the group chat.

Do you you mean the groupchat manager ends up with duplicate content?

Mainly because agent_spec.config.system_message = agent_spec.config.system_message + "\n\n" + skills_prompt. When sanitize_agent_spec is called externally many times, skills will be added to system Prompts many times. if skills exist.

victordibia commented 5 months ago

@cillyfly ,

Thanks for reporting this. On further inspection, it looks like the issue is that self.sanitize_agent_spec(agent_config) is needlessly called in self.load(self.sanitize_agent_spec(agent_config)) as sanitize_agent is already called in load. This should be just self.load(self.agent_config).

This has been fixed in the updated version

pip install -U autogenstudio