boto / boto3

AWS SDK for Python
https://aws.amazon.com/sdk-for-python/
Apache License 2.0
8.92k stars 1.85k forks source link

'Contents' key in S3 paginator's page is assumed to be there #4183

Closed cristiangiann closed 4 weeks ago

cristiangiann commented 1 month ago

Describe the issue

From Boto3 documentation, Page iterators' pages seems to be expected to contain the "Contents" key, which is not always true.

From the following code snippet, similar to the one specified in boto3 paginators' documentation (https://boto3.amazonaws.com/v1/documentation/api/latest/guide/paginators.html#filtering-results), I got a KeyError due to the key "Contents" not being present in the page:

    client_s3 = boto3.client('s3', region_name=aws_region)
    paginator = client_s3.get_paginator('list_objects_v2')
    pages = paginator.paginate(Bucket=bucket, Prefix=PREFIX)
    for page in pages:
        for file in page['Contents']:
            ...

I expected the key to be always present, attached to an empty list value if no file matches the criteria, as I haven't found any mention to its absence in the documentation. I'm now wondering if it the goal of the documentation is to show very simple example to readers about how the code works, or to provide all the tools to help developers avoid edge cases.

I'm happy to read more if any other official documentation is present somewhere else.

Links

https://boto3.amazonaws.com/v1/documentation/api/latest/guide/paginators.html

RyanFitzSimmonsAK commented 1 month ago

Hi @cristiangiann, thanks for reaching out. I'm not sure I agree that this is an unusual interaction; a lot of list operations won't return their top level response element (Contents in this case) if nothing matches. The same thing will happen if you do a normal list-objects-v2 request on an empty bucket. I think mentioning this nuance would be a bit out of place on a piece of documentation about paginators, since it's coming from a general AWS behavior. Thoughts?

github-actions[bot] commented 1 month ago

Greetings! It looks like this issue hasn’t been active in longer than five days. We encourage you to check if this is still an issue in the latest release. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or upvote with a reaction on the initial post to prevent automatic closure. If the issue is already closed, please feel free to open a new one.