A question about stream.read()

[llnancy]

        # get object from s3
        response = await client.get_object(Bucket=bucket, Key=key)
        # this will ensure the connection is correctly re-used/closed
        async with response['Body'] as stream:
            assert await stream.read() == data

For large files (more than GB), stream.read() will cause OOM. Are there any examples of segmented reading?

[jakob-keller]

Awaiting stream.read() will read the whole response’s body as bytes, potentially resulting in OOM issues.

You may want to look at stream.content and the StreamReader reading methods.

I hope this points you in the right direction.

[thehesiod]

there's also the StreamingBody helpers: https://github.com/aio-libs/aiobotocore/blob/9c6db8a9fd12a244a0b7413fd182857e603d08f9/aiobotocore/response.py#L19 which mimics the version from botocore available via response['Body']