Open vijay-cohere opened 3 months ago
WebArena websites require authentication before being used. The README in Webarena specifies how to setup authentication so you end up with an .auth
folder with cookies with login information. I have copied over the instruction below
mkdir -p ./.auth
python browser_env/auto_login.py
Once you have the .auth
folder, copy it over to webagents-step/
root directory. You should then be signed in to all the websites.
(I have the updated the README with these instructions)
When I run SteP with this code on my environment, I'm seeing significantly worse performance than reported in the paper. I was digging into this and I see sign-in pages in the environment state that I don't see when running the "run.py" script in WebArena, e.g.:
This looks like the Gitlab sign-in page. In the default WebArena runner script, you never see this, because cookies are loaded from file to avoid sign-in.
I think the WebArena environment in this repo is broken because it's missing this cookie loading.