aio-libs / aiobotocore

asyncio support for botocore library using aiohttp
https://aiobotocore.rtfd.io
Apache License 2.0
1.14k stars 179 forks source link

Unclosed coonnection #1124

Open maretodoric opened 1 month ago

maretodoric commented 1 month ago

Describe the bug

Unclosed connection
client_connection: Connection<ConnectionKey(host='bucket-name.s3.eu-central-1.amazonaws.com', port=443, is_ssl=True, ssl=None, proxy=None, proxy_auth=None, proxy_headers_hash=None)>

Warning shows up when using S3 client, get_object method.

This is the code I've used:

async def get_s3_object(bucket: str, filepath: str):
    filepath = str(filepath)
    cfg = load_settings()

    session = get_session()
    async with session.create_client('s3', region_name='eu-central-1',
                                     aws_secret_access_key=cfg.aws.secret_key,
                                     aws_access_key_id=cfg.aws.key) as client:
        # get object from s3
        try: s3_ob = await client.get_object(Bucket=bucket, Key=filepath)
        except Exception as e:
            if type(e).__name__ == 'NoSuchKey':
                raise FileNotFoundError(f"File '{basename(filepath)}' not found in bucket: '{bucket}'.")
            else:
                raise
        if s3_ob['ResponseMetadata']['HTTPStatusCode'] != 200:
            raise Exception(f"HTTP Status Code invalid. Response: {s3_ob}")
        else:
            return S3Object(bucket, filepath, s3_ob['ContentLength'], s3_ob['ContentType'])

Checklist

pip freeze results

aioboto3==12.4.0
aiobotocore==2.12.3
aiofiles==22.1.0
aiohttp==3.8.4
aioitertools==0.11.0
aiomysql==0.2.0
aiosignal==1.3.1
alembic==1.11.3
anyio==3.7.1
async-timeout==4.0.2
asyncssh==2.13.2
attrs==23.1.0
beautifulsoup4==4.12.2
boto3==1.34.69
botocore==1.34.69
certifi==2023.5.7
cffi==1.15.1
charset-normalizer==2.1.1
click==8.1.4
cryptography==39.0.0
dccalertmanager @ git+https://dcc-gitlab.addiko.com/mtodoric/pydccalertmanager.git@c88cd8af2415394621b10781d693ac2f52022bb8
dccjenkins @ git+https://dcc-gitlab.addiko.com/mtodoric/pydccjenkins.git@befd2d732e2f633e7965d11daa58af83003a7a3e
dccpassbolt @ git+https://dcc-gitlab.addiko.com/mtodoric/pydccpassbolt.git@ee8fd08e569ebc7a5a68e230801fa2c9763ea9b9
dccwebtool @ file:///home/mtodoric/dev-in-progress/pydccwebtool
dnspython==2.0.0
ecs-logging==2.2.0
elastic-apm==6.22.3
elasticsearch==7.17.6
elasticsearch-dsl==7.4.0
exceptiongroup==1.1.2
fastapi==0.90.0
frozenlist==1.3.3
gitdb==4.0.10
GitPython==3.1.32
greenlet==2.0.2
h11==0.14.0
httpcore==0.16.3
httpx==0.23.3
idna==3.4
iniconfig==2.0.0
ipwhois==1.2.0
Jinja2==3.1.2
jmespath==1.0.1
Mako==1.2.4
MarkupSafe==2.1.3
multidict==6.0.4
packaging==23.1
pluggy==1.2.0
pycparser==2.21
pydantic==1.10.9
PyMySQL==1.1.0
pytest==7.4.0
python-dateutil==2.8.2
python-dotenv==1.0.1
python-gnupg==0.5.1
python-multipart==0.0.5
PyYAML==6.0
requests==2.28.1
rfc3986==1.5.0
s3transfer==0.10.1
six==1.16.0
SkPy==0.10.5
smmap==5.0.0
sniffio==1.3.0
soupsieve==2.4.1
SQLAlchemy==2.0.15
starlette==0.22.0
tomli==2.0.1
typing_extensions==4.7.1
urllib3==1.26.16
uvicorn==0.20.0
websockets==10.4
wrapt==1.16.0
yarl==1.9.2

Environment:

Additional context I've first noticed this when using aioboto3, however when i was told to try and replicate this with aiobotocore - i did and it can be replicated. Warning itself doesn't appear to cause any issues but potential high number of unclosed connections could pose an issue.