Async multiplex tending to be alot slower than sync multiplex

Summary.

I ran a simple http request call's to benchmark various wrappers that Niquests provided, where server supports h2 protocol. The async version with multiplex is almost 5-6 times slower compared to sync version with multiplex. Would request for help if this could be done a better way.

Expected Result - N/A

What you expected. - Usually Async version will be quite faster than sync version, when performing multiple network requests. Atleast, thats what I been observed with async/await calls with http1.1.

Actual Result: Here the Async multiplex was 5-6 times slower when compared to Sync multiplex calls.

What happened instead.

Reproduction Steps

from niquests import Session, AsyncSession
from asyncio import run as asyncio_run
from asyncio import gather as asyncio_gather
from aiohttp import ClientSession

#: You can adjust it as you want and verify the multiplexed advantage!
REQUEST_COUNT = 20
REQUEST_URL = "https://documenter.getpostman.com/view/4858910/S1LpZrgg"

def make_requests(url: str, count: int, use_multiplexed: bool):
    before = time()
    responses = []
    with Session(multiplexed=use_multiplexed) as s:
        for _ in range(count):
            responses.append(s.get(url))
            # print(f"request {_+1}...OK", end=", ", flush=True)
    end = time()
    print([r.status_code for r in responses], end=", ")
    print()
    print(f"{end - before} seconds elapsed ({'multiplexed' if use_multiplexed else 'standard'})")

async def async_make_requests(url: str, count: int, use_multiplexed: bool):
    before = time()
    responses = []
    async with AsyncSession(multiplexed=use_multiplexed) as s:
        for _ in range(count):
            responses.append(await s.get(url))
            # print(f"request {_+1}...OK", end=", ", flush=True)
        await s.gather()
    end = time()
    print([r.status_code for r in responses], end=",")
    print()
    print(f"{end - before} seconds elapsed ({'multiplexed' if use_multiplexed else 'standard'})", end=",")
    print()

async def aio_each_session(url: str, session: ClientSession):
    async with session.get(url) as res:
        r = res.status
        await res.text()
        return r

async def aio_make_requessts(url: str, count: int):
    before = time()
    responses = []
    async with ClientSession() as s:
        for _ in range(count):
            responses.append(aio_each_session(url=url, session=s))
        res = await asyncio_gather(*responses)
    end = time()
    print([r for r in res], end=",")
    print()
    print(f"{end - before} seconds elapsed ('standard')", end=",")
    print()

async def _main():
    #: Let's start with the same good old request one request at a time.
    print("-------------------Sync---------------")
    print("> Without multiplexing:")
    make_requests(REQUEST_URL, REQUEST_COUNT, False)
    #: Now we'll take advantage of a multiplexed connection.
    print("> With multiplexing:")
    make_requests(REQUEST_URL, REQUEST_COUNT, True)
    print("---------------------------Async--------------------")
    print("> Without multiplexing:")
    await async_make_requests(REQUEST_URL, REQUEST_COUNT, False)
    #: Now we'll take advantage of a multiplexed connection.
    print("> With multiplexing:")
    await async_make_requests(REQUEST_URL, REQUEST_COUNT, True)
    print("------------------aiohttp--------------------------")
    await aio_make_requessts(url=REQUEST_URL, count=REQUEST_COUNT)

    print("------------------aiohttp--------------------------")
    await aio_make_requessts(url=REQUEST_URL, count=REQUEST_COUNT)
    print("---------------------------Async--------------------")
    print("> Without multiplexing:")
    await async_make_requests(REQUEST_URL, REQUEST_COUNT, False)
    #: Now we'll take advantage of a multiplexed connection.
    print("> With multiplexing:")
    await async_make_requests(REQUEST_URL, REQUEST_COUNT, True)

    #: Let's start with the same good old request one request at a time.
    print("-------------------Sync---------------")
    print("> Without multiplexing:")
    make_requests(REQUEST_URL, REQUEST_COUNT, False)
    #: Now we'll take advantage of a multiplexed connection.
    print("> With multiplexing:")
    make_requests(REQUEST_URL, REQUEST_COUNT, True)

asyncio_run(_main())

System Information

$ python -m niquests.help

{
  "charset_normalizer": {
    "version": "3.3.2"
  },
  "http1": {
    "h11": "0.14.0"
  },
  "http2": {
    "jh2": "5.0.3"
  },
  "http3": {
    "enabled": true,
    "qh3": "1.1.0"
  },
  "idna": {
    "version": "3.10"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.12.4"
  },
  "niquests": {
    "version": "3.8.0"
  },
  "ocsp": {
    "enabled": true
  },
  "platform": {
    "release": "10",
    "system": "Windows"
  },
  "system_ssl": {
    "version": "300000d0"
  },
  "urllib3.future": {
    "cohabitation_version": null,
    "version": "2.9.900"
  },
  "wassima": {
    "certifi_fallback": false,
    "enabled": true,
    "version": "1.1.2"
  }
}

Results obtained -

Summary - I did requests call in following order and one more requests in reverse order as shared below, just to make sure last wrappers making the requests are not receiving requests from throttling CPU. Turns out its not the case, as the time was quite comparable making irrespective of the order.

Niquests

Sync calls (Standard) - 7.295527696609497 seconds elapsed (standard)
Sync call (with Multiplex) - 0.2068800926208496 seconds elapsed (multiplexed)
Sync calls (Standard) - 6.933176517486572 seconds elapsed (standard)
Async calls (with Multiplex) - 1.4766261577606201 seconds elapsed (multiplexed)

Aiohttp
1. Async call (Standard) - 2.1907169818878174 seconds elapsed ('standard')

Total output obtained

-------------------Sync---------------
> Without multiplexing:
[200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200], 
7.295527696609497 seconds elapsed (standard)
> With multiplexing:
[200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200], 
0.2068800926208496 seconds elapsed (multiplexed)
---------------------------Async--------------------
> Without multiplexing:
[200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200],
6.933176517486572 seconds elapsed (standard),
> With multiplexing:
[200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200],
1.4766261577606201 seconds elapsed (multiplexed),  
------------------aiohttp--------------------------
[200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200],
2.1907169818878174 seconds elapsed ('standard'),   
------------------aiohttp--------------------------
[200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200],
2.2599029541015625 seconds elapsed ('standard'),
---------------------------Async--------------------
> Without multiplexing:
[200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200],
5.6602489948272705 seconds elapsed (standard),
> With multiplexing:
[200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200],
1.3796124458312988 seconds elapsed (multiplexed),
-------------------Sync---------------
> Without multiplexing:
[429, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200], 
5.500334024429321 seconds elapsed (standard)
> With multiplexing:
[429, 429, 429, 429, 429, 429, 429, 429, 429, 429, 429, 429, 429, 429, 429, 429, 429, 429, 429, 429], 
0.2084817886352539 seconds elapsed (multiplexed)

Also want to say I did discover Niquests yesterday, it is a beautiful gift for python. And also would requests for help in integrating this for Fastapi.

Thank you.

jawah / niquests