web-arena-x / webarena

Code repo for "WebArena: A Realistic Web Environment for Building Autonomous Agents"
https://webarena.dev
Apache License 2.0
632 stars 90 forks source link

Problem when redirecting to a new page in a new tab #117

Open taogoddd opened 3 months ago

taogoddd commented 3 months ago

Dear authors,

Thanks for your brilliant work. When I tried to use this benchmark, I found two possible problems:

  1. if some action, e.g. clicking a link, will open a new page in a new tab (instead of redirecting to a page in the same tab), the "page" variable of the Env class will still be the same page (rather than the newly opened page), which means the page that is visible to the agent will still be the same page. This is a bit unreasonable because the agent will not realize the existence of the new page and just keep doing the same action on the same page.
  2. Given the first problem, there are two possible solutions: 1. assign the new page in the new tab to the "page" variable when a new tab is created, i.e. let the current page be the newly opened page; 2. tell the existence of the new page to the agent and also provide a "switch_tab" action in the action space to enable it to switch to the new tab. I think the former solution is more reasonable because it is more like the real situation. Also, "switch_tab" should also be added (it seems not in the current action space) anyway because some tasks involve multi-tab, and the agent can not do the task without the ability to switch between tabs.

Please tell me if anything is missing here. Thanks in advance!

taogoddd commented 3 months ago

One example of opening a new page in a new tab is when the agent tries to click the "Go to Advanced Reporting" button in the shopping admin website like this:

image
shuyanzhou commented 3 months ago

Thank you very much, we are fixing this issue #102