1K Request can't be finished in a simultaneous way in aiohttp

[KevinZhou92]

Long story short

I set up a server for nlu service use Klein, and i was trying some pressure test using aiohttp. However, the maximum simultaneous seems to be under 100. If i have the number as 1000, then i will receive many ConnectionRefusedErrors.

I also tried another pressure test script using multiproccessing and it worked well, so i'm quite confusing about which part of aiohttp is restricting the performance?

aiohttp Code

import time
import asyncio, json
from aiohttp import ClientSession
from aiohttp.connector import TCPConnector, BaseConnector

payload = json.dumps({
    "model_name": ["NCC_NOSE_RS_WFSB_YM","PTP_NPN"],
    "text": "Hello",
    "version_id": "1"})

async def fetch(url):
    async with ClientSession() as session:
        # try:
        async with session.post(url, data=payload) as response:
            return await response.read()
        # except:   
        #     print('unable to connect to device')

async def run(loop,  r):
    url = "http://localhost:8081/predict"
    tasks = []
    for i in range(r):
        task = asyncio.ensure_future(fetch(url.format(i)))
        tasks.append(task)
    responses = await asyncio.gather(*tasks)
       # you now have all response bodies in this variable
    #print(responses)

def print_responses(result):
    print(result)

start = time.time()
loop = asyncio.get_event_loop()
future = asyncio.ensure_future(run(loop, 500))
loop.run_until_complete(future)
print(time.time() -start)

multiprocessing Code

import requests
import multiprocessing
import time
import json
import sys

def pressure_test():
    url = "http://localhost:8081/predict"
    payload = json.dumps({
        "model_name": ["NCC_NOSE_RS_WFSB_YM","PTP_NPN"],
        "text": "hello",
        "version_id": "1"})
    a = time.time()
    response = requests.post(url, data=payload)
    print(response.content)
    b = time.time()
    delta = b-a

if __name__ == "__main__": 
  pool = multiprocessing.Pool(processes=10)
  a = time.time()
  for i in range(500):
      pool.apply_async(pressure_test,())
  pool.close()
  pool.join()
  b = time.time()
  c = b-a
  print(str(c)) 

Error Log

Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 822, in _wrap_create_connection return await self._loop.create_connection(*args, **kwargs) File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 778, in create_connection raise exceptions[0] File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 765, in create_connection yield from self.sock_connect(sock, address) File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/selector_events.py", line 450, in sock_connect return (yield from fut) File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/selector_events.py", line 480, in _sock_connect_cb raise OSError(err, 'Connect call failed %s' % (address,)) ConnectionRefusedError: [Errno 61] Connect call failed ('::1', 8081)

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "test/pressure_test.py", line 37, in loop.run_until_complete(future) File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 468, in run_until_complete return future.result() File "test/pressure_test.py", line 26, in run responses = await asyncio.gather(*tasks) File "test/pressure_test.py", line 15, in fetch async with session.post(url, data=payload) as response: File "/usr/local/lib/python3.6/site-packages/aiohttp/client.py", line 855, in aenter self._resp = await self._coro File "/usr/local/lib/python3.6/site-packages/aiohttp/client.py", line 370, in _request timeout=timeout File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 445, in connect proto = await self._create_connection(req, traces, timeout) File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 757, in _create_connection req, traces, timeout) File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 879, in _create_direct_connection raise last_exc File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 862, in _create_direct_connection req=req, client_error=client_error) File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 829, in _wrap_create_connection raise client_error(req.connection_key, exc) from exc aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host localhost:8081 ssl:None [Connect call failed ('::1', 8081)]

Your environment

aiohttp 3.4.4 python 3.6.5 MacOS 10.13.6

[asvetlov]

GitMate.io thinks possibly related issues are https://github.com/aio-libs/aiohttp/issues/2920 (AIOHttp failing after some requests), https://github.com/aio-libs/aiohttp/issues/660 (aiohttp.request hangs on some URLs), https://github.com/aio-libs/aiohttp/issues/206 (SSL issue with aiohttp.request), https://github.com/aio-libs/aiohttp/issues/2540 (Drop await aiohttp.request(...)), and https://github.com/aio-libs/aiohttp/issues/127 (verify_ssl for aiohttp.request).

[asvetlov]

Looking on the stack trace I see that on your box Python is compiled with IPv6, OS declared to support it but actual call fails.

Replacement localhost with 127.0.0.1 can help.

[KevinZhou92]

@asvetlov Thanks for replying Andrew. I changed the address to 127.0.0.1. Then i encountered a ConnectionResetError.

Error Log

Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 822, in _wrap_create_connection return await self._loop.create_connection(*args, **kwargs) File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 778, in create_connection raise exceptions[0] File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 765, in create_connection yield from self.sock_connect(sock, address) File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/selector_events.py", line 450, in sock_connect return (yield from fut) File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/selector_events.py", line 480, in _sock_connect_cb raise OSError(err, 'Connect call failed %s' % (address,)) ConnectionResetError: [Errno 54] Connect call failed ('127.0.0.1', 8081)

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "test/pressure_test.py", line 37, in loop.run_until_complete(future) File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 468, in run_until_complete return future.result() File "test/pressure_test.py", line 26, in run responses = await asyncio.gather(*tasks) File "test/pressure_test.py", line 15, in fetch async with session.post(url, data=payload) as response: File "/usr/local/lib/python3.6/site-packages/aiohttp/client.py", line 855, in aenter self._resp = await self._coro File "/usr/local/lib/python3.6/site-packages/aiohttp/client.py", line 370, in _request timeout=timeout File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 445, in connect proto = await self._create_connection(req, traces, timeout) File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 757, in _create_connection req, traces, timeout) File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 879, in _create_direct_connection raise last_exc File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 862, in _create_direct_connection req=req, client_error=client_error) File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 829, in _wrap_create_connection raise client_error(req.connection_key, exc) from exc aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host 127.0.0.1:8081 ssl:None [Connect call failed ('127.0.0.1', 8081)]

[asvetlov]

The difference is: in the asyncio example you use 500 concurrent connections.

The multiprocessed code starts 10 processes for processing 500 requests each one-by-one. As the result, you have concurrency factor 10.

Looks like the server just doesn't support high parallelism.

[derlih]

@asvetlov should we close this issue?