search

web-arena-x / webarena

Code repo for "WebArena: A Realistic Web Environment for Building Autonomous Agents"
https://webarena.dev
Apache License 2.0
647 stars 94 forks source link

A probability that the execution will fail? #65

Closed Yangyi-Chen closed 7 months ago

Yangyi-Chen commented 8 months ago

Thanks for the great work!

I'm wondering whether there is a probability that the execution of taking actions can fail in the environment.

For example, I'm playing with the shopping_admin environment. In the first step, I perform the click action in a certain element: create_id_based_action(f"click [877]").

Sometimes it works and navigates to a new page, but sometimes it doesn't. So is this a common issue in the environment, or maybe just errors on my side? Thanks!

shuyanzhou commented 8 months ago

Could you recall what is the element of 877? Perhaps it is due to not wait long enough and the new page is not loaded yet?

Yangyi-Chen commented 8 months ago

oh i see. That could be a reason.

I also find that a potential reason is the element ID for the same web component (like a button) is not fixed when I load the same web page. Is this can happen sometimes in the environment? Thanks!

shuyanzhou commented 7 months ago

I also find that a potential reason is the element ID for the same web component (like a button) is not fixed when I load the same web page. Is this can happen sometimes in the environment? Thanks!

This is expected. The ID is generated randomly every time.

Yangyi-Chen commented 7 months ago

I see. Thanks for your responses!