ltzheng / Synapse

[ICLR 2024] Trajectory-as-Exemplar Prompting with Memory for Computer Control
https://ltzheng.github.io/Synapse/
MIT License
48 stars 9 forks source link

Error while running miniwob on Windows #5

Closed ARajgor closed 7 months ago

ARajgor commented 8 months ago

While running python run_miniwob.py --env_name book-flight I got errors and found some bugs.

1) Chrome instance is initialized, it shows on the taskbar but is not visible on the screen. That is because of the window-position parameter on instance.py file. for 15-inch screens, it is placed outside the screen. https://github.com/ltzheng/Synapse/blob/81dca4f4600294e2b66a1a0dd49562b187a0dbcb/synapse/envs/miniwob/instance.py#L172-L175

2) There is an extra miniwob in base_url. I tried to print base_url and final self.URL. That's why I got the below error. When you try to run book-flight task it goes to the else part of the subdomain check and extra miniwob is added via "miniwob/{subdomain}.html". while the actual path is Synapse\synapse\envs\miniwob\html\miniwob\book-flight.html

https://github.com/ltzheng/Synapse/blob/81dca4f4600294e2b66a1a0dd49562b187a0dbcb/synapse/envs/miniwob/instance.py#L86 https://github.com/ltzheng/Synapse/blob/81dca4f4600294e2b66a1a0dd49562b187a0dbcb/synapse/envs/miniwob/instance.py#L100 image printing both base_url and self.URL.

image

DevTools listening on ws://127.0.0.1:63662/devtools/browser/8819b473-7ca1-4368-9b51-417d1195151a
ERROR:root:Page did not load properly. Wrong MINIWOB_BASE_URL?
Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\Users\ayush\AppData\Local\Programs\Python\Python311\Lib\threading.py", line 1038, in _bootstrap_inner
    self.run()
  File "D:\D drive\Miraiminds\Synapse\synapse\envs\miniwob\instance.py", line 127, in run
    self.create_driver()
  File "D:\D drive\Miraiminds\Synapse\synapse\envs\miniwob\instance.py", line 199, in create_driver
    raise e
  File "D:\D drive\Miraiminds\Synapse\synapse\envs\miniwob\instance.py", line 194, in create_driver
    WebDriverWait(self.driver, 5).until(
  File "D:\D drive\Miraiminds\Synapse\venv\Lib\site-packages\selenium\webdriver\support\wait.py", line 95, in until
    raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: 
Stacktrace:
        GetHandleVerifier [0x00007FF7F9DF5E42+3538674]
        (No symbol) [0x00007FF7F9A14C02]
        (No symbol) [0x00007FF7F98C5AEB]
        (No symbol) [0x00007FF7F990BF4E]
        (No symbol) [0x00007FF7F990C0CC]
        (No symbol) [0x00007FF7F994E477]
        (No symbol) [0x00007FF7F992F0EF]
        (No symbol) [0x00007FF7F994BDE3]
        (No symbol) [0x00007FF7F992EE53]
        (No symbol) [0x00007FF7F98FF514]
        (No symbol) [0x00007FF7F9900631]
        GetHandleVerifier [0x00007FF7F9E26CAD+3738973]
        GetHandleVerifier [0x00007FF7F9E7C506+4089270]
        GetHandleVerifier [0x00007FF7F9E74823+4057299]
        GetHandleVerifier [0x00007FF7F9B45C49+720121]
        (No symbol) [0x00007FF7F9A2126F]
        (No symbol) [0x00007FF7F9A1C304]
        (No symbol) [0x00007FF7F9A1C432]
        (No symbol) [0x00007FF7F9A0BD04]
        BaseThreadInitThunk [0x00007FFFA499257D+29]
        RtlUserThreadStart [0x00007FFFA5B4AA58+40]

After fixing all the things it working fine. 1) comment out that window-position code.

# options.add_argument(
#     "window-position={},{}".format(
#         9000, 30 + self.index * (self.window_height + 30)
#     )
# )

2) self.url = urlparse.urljoin(base_url, "{}.html".format(subdomain))

Running:

image

ltzheng commented 8 months ago

Thanks for the great catch! We indeed did not test on Windows. Would you mind creating a PR?

ARajgor commented 8 months ago

Absolutely. I will test all the things and also check it on MacOS.