'Unclosed connection' warning when using S3 client

[maretodoric]

• Async AWS SDK for Python version: aioboto3==12.4.0
• Python version: 3.10.12
• Operating System: Ubuntu 22.04.2 LTS (Jammy Jellyfish)

Description

I'm receiving warning when using S3 client:

Unclosed connection
client_connection: Connection<ConnectionKey(host='bucket-name.s3.eu-central-1.amazonaws.com', port=443, is_ssl=True, ssl=None, proxy=None, proxy_auth=None, proxy_headers_hash=None)>

I have following function that is supposed to return information about object stored in S3:

async def get_s3_object(bucket: str, filepath: str):
    cfg = load_settings()
    session  = aioboto3.Session(aws_access_key_id=cfg.aws.key, aws_secret_access_key=cfg.aws.secret_key, region_name=cfg.aws.region)

    async with session.client('s3') as s3:
        try:
            s3_ob = await s3.get_object(Bucket=bucket, Key=filepath)
        except Exception as e:
            if type(e).__name__ == 'NoSuchKey':
                raise FileNotFoundError(f"File '{basename(filepath)}' not found in bucket: '{bucket}'.")
            else:
                raise
        if s3_ob['ResponseMetadata']['HTTPStatusCode'] != 200:
            raise Exception(f"HTTP Status Code invalid. Response: {s3_ob}")
        else:
            return s3_ob

Another function calls get_s3_object and everything works fine, but i get warning mentioned above.

I'm using the client as suggested, by creating a session then using the .client or .resource as context manager, but it appears as if it's not closing the S3 connection

[terricain]

Can you replicate this using aiobotocore alone? I have a suspicion its something to do with how the connection pool will keep connections open to speed up subsequent requests, whilst it's annoying I dont have much time to work on fixing it as it shouldn't affect operation.

[maretodoric]

Hm.. Good one.. It appears i can replicate using aiobotocore.

async def get_s3_object(bucket: str, filepath: str):
    filepath = str(filepath)
    cfg = load_settings()

    session = get_session()
    async with session.create_client('s3', region_name='eu-central-1',
                                     aws_secret_access_key=cfg.aws.secret_key,
                                     aws_access_key_id=cfg.aws.key) as client:
        # get object from s3
        try: s3_ob = await client.get_object(Bucket=bucket, Key=filepath)
        except Exception as e:
            if type(e).__name__ == 'NoSuchKey':
                raise FileNotFoundError(f"File '{basename(filepath)}' not found in bucket: '{bucket}'.")
            else:
                raise
        if s3_ob['ResponseMetadata']['HTTPStatusCode'] != 200:
            raise Exception(f"HTTP Status Code invalid. Response: {s3_ob}")
        else:
            return S3Object(bucket, filepath, s3_ob['ContentLength'], s3_ob['ContentType'])

The get_session is an import from aiobotocore.session

[terricain]

I'd suggest raising an issue over on aiobotocore as most likely the error is benign and should be silenced.

[maretodoric]

Yup, already did. Thanks a lot! :)