search

aio-libs / aiohttp

Asynchronous HTTP client/server framework for asyncio and Python
https://docs.aiohttp.org
Other
15.11k stars 2.02k forks source link

asyncio http call times out #3625

Closed rickcoup closed 1 month ago

rickcoup commented 5 years ago

Long story short

We are making async call to the a site. It times out on it. With the network sniffer, it shows the site responses within 400ms.

Expected behaviour

Get the response back.

Actual behaviour

Throwing the error:

File \"/opt/app-root/src/pppLib.py\", line 407, in fetch\n async with session.post(url, data=pData, headers=head, timeout=timeout) as response:\n File \"/opt/app-root/lib/python3.6/site-packages/aiohttp/client.py\", line 1005, in aenter\n self._resp = await self._coro\n File \"/opt/app-root/lib/python3.6/site-packages/aiohttp/client.py\", line 575, in _request\n break\n File \"/opt/app-root/lib/python3.6/site-packages/aiohttp/helpers.py\", line 585, in exit\n raise asyncio.TimeoutError from None"}   I've checked the client.py code, haven't get much clue on it yet.

Steps to reproduce

The code is like this:

async def fetch(session, url):
    #async with session.get(url) as response:
    async with getattr(session,"get")(url) as response:
        return await response.text()

Your environment

redhat 3.9, python 3.6.3, aiohttp 3.5.4 and asyncio 3.4.3

aio-libs-bot commented 5 years ago

GitMate.io thinks the contributor most likely able to help you is @asvetlov.

Possibly related issues are https://github.com/aio-libs/aiohttp/issues/33 (HTTPS Support), https://github.com/aio-libs/aiohttp/issues/1745 (Call https website through proxy will cause error), https://github.com/aio-libs/aiohttp/issues/320 (HTTP/2 support), https://github.com/aio-libs/aiohttp/issues/858 (replace http parser?), and https://github.com/aio-libs/aiohttp/issues/6 (HTTP pipelining support improvements).

rickcoup commented 5 years ago

The proxy is not an issue. We got response back in normal situation. It happens under heavy number of requests only.

odysseusmax commented 5 years ago

i too get this error 

Traceback (most recent call last):
   File "/app/downloads.py", line 127, in down_File
     async for chunk in r.content.iter_chunked(1024*512):
   File "/app/.heroku/python/lib/python3.7/site-packages/aiohttp/streams.py", line 40, in __anext__
     rv = await self.read_func()
   File "/app/.heroku/python/lib/python3.7/site-packages/aiohttp/streams.py", line 369, in read
    await self._wait('read')
   File "/app/.heroku/python/lib/python3.7/site-packages/aiohttp/streams.py", line 297, in _wait
     await waiter
   File "/app/.heroku/python/lib/python3.7/site-packages/aiohttp/helpers.py", line 585, in __exit__
     raise asyncio.TimeoutError from None
 concurrent.futures._base.TimeoutError
KolbyML commented 5 years ago

so what should you limit the requests to be if its "heavy requests"

KolbyML commented 5 years ago

ya as this issue only affects one object in my scheduler with the proxy i use but not others so 🤔

Dreamsorcerer commented 2 months ago

There's not enough information here to provide any help. It could be that you've set the timeout too low or created too many tasks. The total timeout includes the time it takes to handle everything, not just the network request itself.