search

HENNGE / arsenic

Async WebDriver implementation for asyncio and asyncio-compatible frameworks
Other
349 stars 52 forks source link

Asyncio TimeoutError #152

Open Tony-MK opened 2 years ago

Tony-MK commented 2 years ago

Hello everyone, I would like to mention a minor issue with arsenic.

When the browser takes up a long period of time to start up and as excepted, a timeout exception will be raised. However, when the current code attempts to handle this exception, the asyncio.futures.TimoutError is used in the code. According to the Python documentation, the exception is not available in asyncio.futures.

Therefore, I searched the python documentation and found that the asyncio.TimeoutError exception is supposed to be utilized to handle such asynchronous timeout errors. I hope the code can be fixed as it may bring issues to python programs, especially if the program uses multiple processes. That is all and thanks for your attention and time.

devl00p commented 1 year ago

Hello ! I saw that error too.

Here is a simple code to reproduce:

import asyncio

from arsenic import get_session, browsers, services

async def launch_browser():
    service = services.Geckodriver()
    browser = browsers.Firefox(
        acceptInsecureCerts=True,
    )

    async with get_session(service, browser) as session:
        await session.get('http://127.0.0.1:8000/index.php', timeout=5)
        page_source = await session.get_page_source()

async def main():
    await asyncio.gather(
        launch_browser(),
    )

asyncio.run(main())

Here the timeout is set to 5 for arsenic. The php script is sleeping for 6 seconds:

<?php
sleep(6);
?>

Traceback:

Traceback (most recent call last):
  File "/tmp/headless/arsenic_single_req.py", line 27, in <module>
    asyncio.run(main())
  File "/usr/lib64/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib64/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/tmp/headless/arsenic_single_req.py", line 23, in main
    await asyncio.gather(
  File "/tmp/headless/arsenic_single_req.py", line 19, in launch_browser
    await session.get('http://127.0.0.1:8000/index.php', timeout=5)
  File "/home/sirius/.local/share/virtualenvs/headless-C7_0qtPd/lib/python3.10/site-packages/arsenic/session.py", line 152, in get
    await self._request(
  File "/home/sirius/.local/share/virtualenvs/headless-C7_0qtPd/lib/python3.10/site-packages/arsenic/session.py", line 40, in _request
    status, data = await self.connection.request(
  File "/home/sirius/.local/share/virtualenvs/headless-C7_0qtPd/lib/python3.10/site-packages/arsenic/connection.py", line 55, in wrapper
    return await asyncio.get_event_loop().create_task(func(*args, **kwargs))
  File "/home/sirius/.local/share/virtualenvs/headless-C7_0qtPd/lib/python3.10/site-packages/arsenic/connection.py", line 103, in request
    async with self.session.request(
  File "/home/sirius/.local/share/virtualenvs/headless-C7_0qtPd/lib64/python3.10/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/sirius/.local/share/virtualenvs/headless-C7_0qtPd/lib64/python3.10/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/sirius/.local/share/virtualenvs/headless-C7_0qtPd/lib64/python3.10/site-packages/aiohttp/client_reqrep.py", line 893, in start
    with self._timer:
  File "/home/sirius/.local/share/virtualenvs/headless-C7_0qtPd/lib64/python3.10/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError

Also that timeout is not raised after the delay given to the timeout argument so if the page waits 30 secs the timeout is raised after 30secs, not 5 secs

Behavior seen:

Expected behavior:

Versions used: