ResourceWarning: unclosed transport when using SSL on server

[romasku]

🐞 Describe the bug Server with ssl enabled do not closes _SSLProtocolTransport properly, which leads to ResourceWarning. This

💡 To Reproduce

Run the following script (requires trustme package).

import asyncio
import ssl
import warnings

import aiohttp
import trustme
from aiohttp import web

# Enable all warnings
warnings.simplefilter('always')

async def run():
    # Prepare ssl
    certificate = trustme.CA().issue_server_cert("localhost", "127.0.0.1", "::1")
    ssl_context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
    certificate.configure_cert(ssl_context)

    # Prepare server
    app = web.Application()

    async def do_nothing(request: web.Request) -> web.Response:
        return web.Response(text="OK")

    app.router.add_routes([web.get("", do_nothing)])

    runner = web.AppRunner(app)
    await runner.setup()
    site = web.TCPSite(runner, "127.0.0.1", 8000, ssl_context=ssl_context)
    await site.start()

    # Do request
    connector = aiohttp.TCPConnector(ssl=False)
    async with aiohttp.ClientSession(connector=connector) as session:
        async with session.get("https://localhost:8000") as resp:
            assert await resp.text() == "OK"

    # Close server
    await runner.cleanup()

if __name__ == "__main__":
    asyncio.run(run())

💡 Expected behavior

No warnings are generated.

📋 Logs/tracebacks

python3.9/asyncio/sslproto.py:320: ResourceWarning: unclosed transport <asyncio.sslproto._SSLProtocolTransport object at 0x107471e80>
  _warn(f"unclosed transport {self!r}", ResourceWarning, source=self)

📋 Your version of the Python

$ python --version
Python 3.9.1

📋 Your version of the aiohttp/yarl/multidict distributions

$ python -m pip show aiohttp
Version: 3.7.3

$ python -m pip show multidict
Version: 5.1.0

$ python -m pip show yarl
Version: 1.6.3

📋 Additional context

As I can see from source code, the reason is that web_protocol.RequestHandler.connection_lost is called when transport is still active because of keepalive. As a workaround, one can set keepalive_timeout to zero (it can be OK for tests): runner = web.AppRunner(app, keepalive_timeout=0).

[webknjaz]

@romasku can you reproduce this on master? I'm almost sure it should be fixed with aiohttp v4

[romasku]

Unfotunately, it is still reproducible on master:

$ python test.py 
/usr/lib/python3.9/asyncio/sslproto.py:320: ResourceWarning: unclosed transport <asyncio.sslproto._SSLProtocolTransport object at 0x7f96c350d880>
  _warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
$ python -m pip show aiohttp
Name: aiohttp
Version: 4.0.0a1

AFAIK on master there is a fix for an issue in the client, while this issue is about a problem in the server part of the library, but maybe I am wrong.

[JimFawkes]

Hello, Is there an update on this issue? We seem to be facing this issue too.

[Dreamsorcerer]

I think this is the issue that has a workaround for aiohttp 3.x is documented at: https://docs.aiohttp.org/en/stable/client_advanced.html?highlight=sleep#graceful-shutdown

[Dreamsorcerer]

Thanks for the reproducer @romasku, that was really helpful in figuring this out finally.