[BUG] Can't execute `open youtube` since merge of planner/nested chat support #61 in commit 284091d

[cedricvidal]

Title

Can't execute open youtube example since merge of #61

Description

Since merge of #61, executing open youtube hangs instead of opening the youtube homepage.

Steps to Reproduce

1. Checkout commit a26b870
2. Execute open youtube in assistant
3. The youtube home page will correctly open
4. Checkout commit 284091d
5. Execute open youtube in assistant
6. The agent hangs with the message bellow

>>> Creating AutogenWrapper with ['user', 'planner_agent', 'browser_nav_agent', 'browser_nav_executor'] and 1000 rounds.
INFO:ae.utils.logger:Using model gpt-4o for AutoGen from the environment variables.
>>> Created browser_nav_executor_agent: <ae.utils.autogen_sequential_function_call.UserProxyAgent_SequentialFunctionExecution object at 0x10ac32210>
INFO:ae.utils.logger:User preferences loaded from: /Users/cv/Documents/Projects/agent-e/Agent-E/ae/user_preferences/user_preferences.txt
INFO:ae.utils.logger:User preferences loaded from: /Users/cv/Documents/Projects/agent-e/Agent-E/ae/user_preferences/user_preferences.txt
INFO:ae.utils.logger:User dir: 
INFO:ae.utils.logger:Processing command: open youtube
INFO:ae.utils.logger:Prompt for command: Execute the user task "open youtube" Current Page: https://www.google.com/
{'openurl': <function openurl at 0x11a30af20>, 'enter_text_and_click': <function enter_text_and_click at 0x11a3384a0>, 'get_dom_with_content_type': <function get_dom_with_content_type at 0x11a339260>, 'click': <function click at 0x11a339300>, 'geturl': <function geturl at 0x11a33ba60>, 'bulk_enter_text': <function bulk_enter_text at 0x11a338680>, 'entertext': <function entertext at 0x11a309760>, 'press_key_combination': <function press_key_combination at 0x11a30a700>, 'extract_text_from_pdf': <function extract_text_from_pdf at 0x11a339ee0>}
user (to planner_agent):

Execute the user task "open youtube" Current Page: https://www.google.com/

--------------------------------------------------------------------------------

Expected Behavior

Open youtube homepage

Actual Behavior

Hangs

Screenshots

If applicable, add screenshots to help explain your problem.

Environment

• OS: OSX
• Browser chrome

[teaxio]

@cedricvidal I pulled the latest from master and I tried open youtube You can see from the recording that it worked. https://github.com/EmergenceAI/Agent-E/assets/70958516/040f23e6-699a-46fe-b9ee-e6e0aecb4aa5

[deepak-akkil]

Hi @cedricvidal , i think i know reason for the issue The issue is likely related to a bug in autogen

As a work around, there is an update in the dependencies in the latest version in master (specifically you need to install the nest-asyncio package). If you have not refreshed the dependencies, could you try it and let us know?

[cedricvidal]

Hum, indeed, I had not refreshed the dependencies, my bad.

That being said, after refreshing everything, I still get the exact same behavior

git reset --hard upstream/master
rm -rf .venv/
uv venv --python 3.11
source .venv/bin/activate
uv pip compile pyproject.toml -o requirements.txt
uv pip install -r requirements.txt
uv pip install -r pyproject.toml --extra dev
playwright install
python -u -m ae.main

And I verified, the nest-asyncio package is installed in my venv