ServiceNow / WorkArena

WorkArena: How Capable are Web Agents at Solving Common Knowledge Work Tasks?
https://servicenow.github.io/WorkArena/
Other
103 stars 6 forks source link

What does teardown() do? #10

Closed rhythmcao closed 4 months ago

rhythmcao commented 4 months ago

Thanks a lot for providing this wonderful benchmark and environment.

Our team are developing based on the BrowserGym environment (trying to wrap over the BrowserEnv class). I wonder what the teardown function actually do for each task type. Is there any problem if we do not call this function after a series actions (w/ or w/o invoking the validate function)? It seems it will influence the history on cloud ServiceNow DB records.

gasse commented 4 months ago

Hi @rhythmcao , thank you for your interest!

If you are developing an agent, the task.teardown() function is not meant to be called directly by you. It will be called by the BrowserEnv environment upon termination of an episode (env.reset() and env.close()).

if you are developing a new task, implementing this method is optional. It gives you the possibility to clean things up once the agent is done interacting with the environment (e.g., deleting temporary data). it you not be relied upon too much though, as if your program terminates in the middle of an episode (agent or environment failure) there is no guarantee this method will be called at all.

Hope this helps