web-arena-x / webarena

Code repo for "WebArena: A Realistic Web Environment for Building Autonomous Agents"
https://webarena.dev
Apache License 2.0
676 stars 103 forks source link

Demo server (gitlab) is down; cannot generate the auth sessions #42

Closed wookayin closed 11 months ago

wookayin commented 11 months ago

Since last week, the demo server hosted on your AWS instance seems to have some problems, so the minimal script does not run.

$ cd webarena && python minimal_example.py

FYI

os.environ["SHOPPING"] = "http://ec2-3-131-244-37.us-east-2.compute.amazonaws.com:7770"                                                                  
os.environ["SHOPPING_ADMIN"] = "http://ec2-3-131-244-37.us-east-2.compute.amazonaws.com:7780/admin"                                                      
os.environ["REDDIT"] = "http://ec2-3-131-244-37.us-east-2.compute.amazonaws.com:9999"                                                                    
os.environ["GITLAB"] = "http://ec2-3-131-244-37.us-east-2.compute.amazonaws.com:8023"                                                                    
os.environ["MAP"] = "http://ec2-3-131-244-37.us-east-2.compute.amazonaws.com:3000"                                                                       
os.environ["WIKIPEDIA"] = "http://ec2-3-131-244-37.us-east-2.compute.amazonaws.com:8888/wikipedia_en_all_maxi_2022-05/A/User:The_other_Kiwix_guy/Landing"
os.environ["HOMEPAGE"] = "PASS"  # The home page is not currently hosted in the demo site                                                                

Error - It fails here:

 p = subprocess.run(
     ["python", "scripts/generate_test_data.py"], capture_output=True,
+    check=True
)

# It will generate individual config file for each test example in config_files
assert os.path.exists("config_files/0.json")

Actually error handling is poorly done, the root cause is .auth generation has failed:

FileNotFoundError: [Errno 2] No such file or directory: './.auth/gitlab_state.json'

Stack trace:

Traceback (most recent call last):                                                                                                      
  File ".../webarena/browser_env/auto_login.py", line 128, in <module>                          
    main()                                                                                                                              
  File "<@beartype(__main__.main) at 0x7fbea390ce00>", line 11, in main                                                                 
  File ".../webarena/browser_env/auto_login.py", line 113, in main                              
    renew_comb(list(sorted(pair)))                                                                                                      
  File "<@beartype(__main__.renew_comb) at 0x7fbef4f7afc0>", line 43, in renew_comb                                                     
  File ".../webarena/browser_env/auto_login.py", line 83, in renew_comb                         
    page.get_by_test_id("username-field").click()                                                                                       
  File "$CONDA_PREFIX/lib/python3.11/site-packages/playwright/sync_api/_generated.py", line 15436, in click         
    self._sync(
  File "$CONDA_PREFIX/lib/python3.11/site-packages/playwright/_impl/_sync_base.py", line 104, in _sync
    return task.result()
           ^^^^^^^^^^^^^
  File "$CONDA_PREFIX/lib/python3.11/site-packages/playwright/_impl/_locator.py", line 148, in click
    return await self._frame.click(self._selector, strict=True, **params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "$CONDA_PREFIX/lib/python3.11/site-packages/playwright/_impl/_frame.py", line 489, in click
    await self._channel.send("click", locals_to_params(locals()))
  File "$CONDA_PREFIX/lib/python3.11/site-packages/playwright/_impl/_connection.py", line 61, in send
    return await self._connection.wrap_api_call(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "$CONDA_PREFIX/lib/python3.11/site-packages/playwright/_impl/_connection.py", line 461, in wrap_api_call
    return await cb()
           ^^^^^^^^^^
  File "$CONDA_PREFIX/lib/python3.11/site-packages/playwright/_impl/_connection.py", line 96, in inner_send
    result = next(iter(done)).result()
             ^^^^^^^^^^^^^^^^^^^^^^^^^
playwright._impl._api_types.TimeoutError: Timeout 30000ms exceeded.

I also tried to use the .auth cookies I had generated before, and then I consistently get 502 Errors from the gitlab server instance.

Other information:

Any help would be appreciated.

shuyanzhou commented 11 months ago

Gitlab is back and the authentication on it should work now.

wookayin commented 11 months ago

The minimal_example.py demo still doesn't run (I tried it few days ago and right now). Currently it blocks forever on browser_env/auto_login.py. Can you run the demo?

  52 │   context = browser.new_context()
> 53 │   page = context.new_page()

~The CI tests pass on Github Actions though, but the test_auth_cookie.py also blocks in my local and remote environments.~