search

VRSEN / agency-swarm

The only reliable agent framework built on top of the latest OpenAI Assistants API.
https://vrsen.github.io/agency-swarm/
MIT License
2.74k stars 727 forks source link

Example code in Colab with default settings and default task gone in a loop burning money #180

Closed vanetreg closed 1 month ago

vanetreg commented 1 month ago

Example Colab code linked from here: https://github.com/VRSEN/agency-swarm with link: https://colab.research.google.com/drive/1qGVyK-vIoxZD0dMrMVqCxCsgL1euMLKj with recommended task: # Please create a social media marketing agency with 3 agents that connects to facebook api

The Gradio app was created properly, but after instructing the CEO with the task, after a while it stuck in a loop (of tool creation by the coder), so after burning 3 USDs I stopped the app and revoked the API key.

So pls. re-check this demo code and improve it to avoid such outcomes :- /

bonk1t commented 1 month ago

Hi @vanetreg,

Thank you for bringing this to our attention and for the detailed feedback! We understand how frustrating this issue can be, and we’re here to help you resolve it.

While occasional looping behavior can happen with some models, it’s quite uncommon with more advanced models like GPT-4o. I was unable to reproduce the issue in Colab on my end.

To help us further investigate and ensure reproducibility, we recommend the following steps:

  1. Set the temperature to 0: This ensures the model behaves deterministically and reduces the likelihood of looping. To do this, you’ll need to clone the agency-swarm repository locally and adjust the temperature settings in the agency_swarm/agency/genesis directory. This includes agents and tools, please search for the word "temperature".
  2. Just in case, set the default agency temperature to 0 when initializing Agency in your code: Agency(..., temperature=0).
  3. Share the exact conversation with the “GenesisCEO” agent: This will allow us to better replicate the issue.

Once we have this information, we can work on implementing appropriate guardrails at the framework level to prevent this type of loop in future cases.

Thank you for your understanding and cooperation!

vanetreg commented 1 month ago

Once we have this information, we can work on implementing appropriate guardrails at the framework level to prevent this type of loop in future cases.

@bonk1t I'm sorry, but I won't use more time and resources on testing this Colab. I'll test the code locally in my IDE where I see more info about classes etc, I only suggest to add some run stopping button to the colab code