web-arena-x / visualwebarena

VisualWebArena is a benchmark for multimodal agents.
https://jykoh.com/vwa
MIT License
232 stars 44 forks source link

PyTest fail on test_click_open_new_tab #28

Open anithselva opened 6 months ago

anithselva commented 6 months ago

Hi There,

I'm running into test failures when I run the pytest test suite.

Here is my error:


tests/test_browser_env/test_script_browser_env.py s.s.......F

============================================================ FAILURES ============================================================
____________________________________________________ test_click_open_new_tab _____________________________________________________

accessibility_tree_current_viewport_script_browser_env = <browser_env.envs.ScriptBrowserEnv object at 0x7f1e5af406d0>

    def test_click_open_new_tab(
        accessibility_tree_current_viewport_script_browser_env: ScriptBrowserEnv,
    ) -> None:
        env = accessibility_tree_current_viewport_script_browser_env
        env.reset()
        env.step(
            create_playwright_action(
                "page.goto('https://www.w3schools.com/jsref/tryit.asp?filename=tryjsref_win_open')"
            )
        )
        obs, *_, info = env.step(
            create_playwright_action(
                'page.frame_locator("iframe[name=\\"iframeResult\\"]").get_by_role("button", name="Try it").click()'
            )
        )
        print("TP")
        print(info["page"].url)
>       assert info["page"].url == "https://www.w3schools.com/"
E       AssertionError: assert 'https://www....sref_win_open' == 'https://www.w3schools.com/'
E         - https://www.w3schools.com/
E         + https://www.w3schools.com/jsref/tryit.asp?filename=tryjsref_win_open

tests/test_browser_env/test_script_browser_env.py:293: AssertionError
------------------------------------------------------ Captured stdout call ------------------------------------------------------
TP
https://www.w3schools.com/jsref/tryit.asp?filename=tryjsref_win_open

I see that there has been some activity this , and that this is actually a new test itself from https://github.com/web-arena-x/visualwebarena/pull/23

How can I resolve this?

Thanks