shuyanzhou
commented
1 year ago Thanks! I also encountered this problem. There might be better solutions, but here is my solution -- I periodically checked whether any trajectory contains the login page (e.g., grep -l "Welcome, please sign in" *.html), if so, I pause the test and re-generate the cookies.
Ideally, we want to reset our environment and cookies after every example; however, it is too time-consuming.
Sometimes when running many tasks, the agent will accidentally click "sign out" halfway through these tasks. All the remaining tasks that require user login are then unable to run.