With regard to environment reset

ServiceNow / WorkArena

WorkArena: How Capable are Web Agents at Solving Common Knowledge Work Tasks?

https://servicenow.github.io/WorkArena/

Other

122 stars 11 forks source link

With regard to environment reset #22

Closed dgjun32 closed 2 months ago

dgjun32 commented 2 months ago

After the entire evaluation, how can I reset the whole environment?

Do I have to release the instance and request a new one? or just run the 'workarena-install' command with existing instance?

aldro61 commented 2 months ago

If you want to completely reset the environment, you need to release your instance, create a new one, and re-run workarena-install.

However, note that this is not necessary. We take measures to keep tasks isolated from one another:

Each task is executed in a new user with a random name (data doesn't transport across users)
Each task has a teardown function that cleans up any data created during the task execution

Hope this helps!

dgjun32 commented 1 month ago

With respect to the teardown function in task, can you specify the usage of the function with more details?

aldro61 commented 1 month ago

This function is called after each episode to make sure the task leaves no trace in the database. It wipes any data that was created to make the task possible (by the setup function) or data created by the agent.