asappresearch / webagents-step

MIT License
36 stars 8 forks source link

Cookies not loaded for website sign-in #2

Open vijay-cohere opened 3 months ago

vijay-cohere commented 3 months ago

When I run SteP with this code on my environment, I'm seeing significantly worse performance than reported in the paper. I was digging into this and I see sign-in pages in the environment state that I don't see when running the "run.py" script in WebArena, e.g.:

[1] RootWebArea 'Sign in · GitLab' focused: True
        [24] img 'GitLab Community Edition'
        [25] heading 'GitLab Community Edition'
        [11] tabpanel ''
                [13] Section '' live: assertive atomic: False relevant: additions text
                        [50] StaticText 'Username or email'
                        [15] textbox 'Username or email' focused: True required: True
                        [56] StaticText 'Password'
                        [54] textbox 'Password' required: True
                        [64] checkbox 'Remember me' checked: false
                        [66] link 'Forgot your password?'
                        [62] button 'Sign in'
        [29] StaticText "Don't have an account yet? "
        [30] link 'Register now'
        [19] separator '' orientation: horizontal
        [31] link 'Explore'
        [33] link 'Help'
        [35] link 'About GitLab'
        [37] link 'Community forum'

This looks like the Gitlab sign-in page. In the default WebArena runner script, you never see this, because cookies are loaded from file to avoid sign-in.

Screenshot 2024-07-01 at 10 47 36 AM

I think the WebArena environment in this repo is broken because it's missing this cookie loading.

psodhi-asapp commented 3 months ago

WebArena websites require authentication before being used. The README in Webarena specifies how to setup authentication so you end up with an .auth folder with cookies with login information. I have copied over the instruction below

mkdir -p ./.auth
python browser_env/auto_login.py

Once you have the .auth folder, copy it over to webagents-step/ root directory. You should then be signed in to all the websites.

(I have the updated the README with these instructions)