search

microsoft / playwright-python

Python version of the Playwright testing and automation library.
https://playwright.dev/python/
Apache License 2.0
11.83k stars 903 forks source link

[BUG] Terminating the Selenium Grid script freezes Playwright #1164

Open Perlence opened 2 years ago

Perlence commented 2 years ago

Context:

Code Snippet

import time
from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(headless=False)
    try:
        page = browser.new_page()
        page.goto('https://en.wikipedia.org')
        print('sleeping...')
        time.sleep(10)
    finally:
        browser.close()

Describe the bug

I'm running a Selenium Grid container:

$ docker ps
CONTAINER ID   IMAGE                                       COMMAND                  CREATED          STATUS          PORTS                                                                                  NAMES
6b005a57908f   selenium/standalone-chrome:4.1.2-20220208   "/opt/bin/entry_poin…"   20 minutes ago   Up 15 minutes   0.0.0.0:4444->4444/tcp, :::4444->4444/tcp, 0.0.0.0:5900->5900/tcp, :::5900->5900/tcp   playground_selenium_1

Sending a SIGINT to the script while it sleeps freezes the process. Sending it again crashes the process with the following:

$ env SELENIUM_REMOTE_URL=http://localhost:4444/wd/hub python playwright_playground.py
sleeping...
^C
^CTraceback (most recent call last):
  File "/home/sviatoslav/Workspace/playground/playwright_playground.py", line 11, in <module>
    time.sleep(10)
KeyboardInterrupt

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/sviatoslav/Workspace/playground/playwright_playground.py", line 13, in <module>
    browser.close()
  File "/home/sviatoslav/Workspace/playground/env/lib/python3.10/site-packages/playwright/sync_api/_generated.py", line 11153, in close
    self._sync("browser.close", self._impl_obj.close())
  File "/home/sviatoslav/Workspace/playground/env/lib/python3.10/site-packages/playwright/_impl/_sync_base.py", line 109, in _sync
    self._dispatcher_fiber.switch()
KeyboardInterrupt
Task was destroyed but it is pending!
task: <Task pending name='Task-7' coro=<Browser.close() done, defined at /home/sviatoslav/Workspace/playground/env/lib/python3.10/site-packages/playwright/_impl/_browser.py:164> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[SyncBase._sync.<locals>.callback() at /home/sviatoslav/Workspace/playground/env/lib/python3.10/site-packages/playwright/_impl/_sync_base.py:104]>

After this the Selenium session is left open. New connections to it fail, the Selenium container usually has to be restarted.

I've experienced this both with the headed and the headless modes. This doesn't happen when I don't use Selenium, i.e. closing the browser that runs natively works fine.

pavelfeldman commented 2 years ago

https://playwright.dev/python/docs/intro#timesleep-leads-to-outdated-state - you can't use time.sleep!

Perlence commented 2 years ago

Sorry, my bad! However, the same happens if I change sleep to wait_for_timeout.

$ env SELENIUM_REMOTE_URL=http://localhost:4444/wd/hub python playwright_playground.py
sleeping...
^C
^CTraceback (most recent call last):
  File "/home/sviatoslav/Workspace/playground/playwright_playground.py", line 9, in <module>
    page.wait_for_timeout(5000)
  File "/home/sviatoslav/Workspace/playground/env/lib/python3.10/site-packages/playwright/sync_api/_generated.py", line 9035, in wait_for_timeout
    self._sync(
  File "/home/sviatoslav/Workspace/playground/env/lib/python3.10/site-packages/playwright/_impl/_sync_base.py", line 109, in _sync
    self._dispatcher_fiber.switch()
  File "/home/sviatoslav/Workspace/playground/env/lib/python3.10/site-packages/playwright/sync_api/_context_manager.py", line 48, in greenlet_main
    loop.run_until_complete(self._connection.run_as_sync())
  File "/home/sviatoslav/.pyenv/versions/3.10.1/lib/python3.10/asyncio/base_events.py", line 628, in run_until_complete
    self.run_forever()
  File "/home/sviatoslav/.pyenv/versions/3.10.1/lib/python3.10/asyncio/base_events.py", line 595, in run_forever
    self._run_once()
  File "/home/sviatoslav/.pyenv/versions/3.10.1/lib/python3.10/asyncio/base_events.py", line 1845, in _run_once
    event_list = self._selector.select(timeout)
  File "/home/sviatoslav/.pyenv/versions/3.10.1/lib/python3.10/selectors.py", line 469, in select
    fd_event_list = self._selector.poll(timeout, max_ev)
KeyboardInterrupt

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/sviatoslav/Workspace/playground/playwright_playground.py", line 11, in <module>
    browser.close()
  File "/home/sviatoslav/Workspace/playground/env/lib/python3.10/site-packages/playwright/sync_api/_generated.py", line 11153, in close
    self._sync("browser.close", self._impl_obj.close())
  File "/home/sviatoslav/Workspace/playground/env/lib/python3.10/site-packages/playwright/_impl/_sync_base.py", line 109, in _sync
    self._dispatcher_fiber.switch()
KeyboardInterrupt
Task was destroyed but it is pending!
task: <Task pending name='Task-8' coro=<Browser.close() running at /home/sviatoslav/Workspace/playground/env/lib/python3.10/site-packages/playwright/_impl/_browser.py:164> cb=[SyncBase._sync.<locals>.callback() at /home/sviatoslav/Workspace/playground/env/lib/python3.10/site-packages/playwright/_impl/_sync_base.py:104]>
/home/sviatoslav/.pyenv/versions/3.10.1/lib/python3.10/asyncio/base_events.py:666: RuntimeWarning: coroutine 'Browser.close' was never awaited
Exception ignored in: <function BaseSubprocessTransport.__del__ at 0x7f16f9e47760>
Traceback (most recent call last):
  File "/home/sviatoslav/.pyenv/versions/3.10.1/lib/python3.10/asyncio/base_subprocess.py", line 126, in __del__
  File "/home/sviatoslav/.pyenv/versions/3.10.1/lib/python3.10/asyncio/base_subprocess.py", line 104, in close
  File "/home/sviatoslav/.pyenv/versions/3.10.1/lib/python3.10/asyncio/unix_events.py", line 546, in close
  File "/home/sviatoslav/.pyenv/versions/3.10.1/lib/python3.10/asyncio/unix_events.py", line 570, in _close
  File "/home/sviatoslav/.pyenv/versions/3.10.1/lib/python3.10/asyncio/base_events.py", line 745, in call_soon
  File "/home/sviatoslav/.pyenv/versions/3.10.1/lib/python3.10/asyncio/base_events.py", line 510, in _check_closed
RuntimeError: Event loop is closed
Task was destroyed but it is pending!
task: <Task pending name='Task-7' coro=<Page.wait_for_timeout() done, defined at /home/sviatoslav/Workspace/playground/env/lib/python3.10/site-packages/playwright/_impl/_page.py:823> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[SyncBase._sync.<locals>.callback() at /home/sviatoslav/Workspace/playground/env/lib/python3.10/site-packages/playwright/_impl/_sync_base.py:104]>
mxschmitt commented 2 years ago

I was able to reproduce it, its not related to Selenium, its related to the sync exception handling.

rmorshea commented 2 years ago

Maybe related to https://github.com/microsoft/playwright-python/issues/818?

DHpie commented 2 years ago

I have the same problem

log-info- [web run] OnAfter run event!
F:\devTools\Python-3.7\lib\site-packages\behave\runner.py:585: RuntimeWarning: coroutine 'Browser.close' was never awaited
  statement.error_message = error_message
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
mxschmitt commented 2 years ago

Sorry for the late reply. Was able to reproduce it without Selenium as well in the sync and async version.

The intended behaviour would be that wait_for_timeout will throw the KeyboardInterrupt exception.

docker run -p 4444:4444 -p 7900:7900 --shm-size="2g" --rm -e SE_OPTS="--host localhost" selenium/standalone-chrome:4.3.0-20220726

Relates https://github.com/microsoft/playwright-python/pull/819/files

ifduyue commented 4 months ago

I've encountered this too