How to get and use the accessibility tree？

web-arena-x / webarena

Code repo for "WebArena: A Realistic Web Environment for Building Autonomous Agents"

https://webarena.dev

Apache License 2.0

700 stars 108 forks source link

Closed HarryZhou-618 closed 8 months ago

HarryZhou-618 commented 8 months ago

Hi there~, your excellent work has provided us with a great web test environment! But I still have some questions about the accessibility tree:

How do we get the accessibility tree through some APIs?
And after we get the accessibility tree, for the elements selected by the LLM, how can we map the element to the DOM elements which we can make actions on them like clicking etc.? Looking forward to your reply!