youtype / mypy_boto3_builder

Type annotations builder for boto3 compatible with VSCode, PyCharm, Emacs, Sublime Text, pyright and mypy.
https://youtype.github.io/mypy_boto3_builder/
MIT License
544 stars 36 forks source link

Incorrect Return Type for `S3.Client.select_object_content` #197

Closed tas12740 closed 1 year ago

tas12740 commented 1 year ago

I am really enjoying this project so far and appreciate all the work that has gone into it! It's providing a lot of utility on one of the projects on which I'm working. With that said, I'm running into an error with a particular S3 call.

Describe the bug As documented in the mypy_boto3_s3 docs, the Payload key of S3.Client.select_object_content returns a TypedDict. As far as I can understand it from the boto3 docs, the Payload key is a botocore.eventstream.EventStream (doc). That is, instead of being a dict, it is an Iterable of TypedDict results.

To Reproduce Steps to reproduce the behavior:

  1. Install boto3-stubs[s3]
  2. Run mypy/pyright on the following code sample
from boto3 import Session

client = Session().client("s3", region_name="us-east-1")

response = client.select_object_content(
    Bucket="my-bucket",
    Key="my/key.csv",
    ExpressionType="SQL",
    # some details elided for conciseness
    Expression="my-sql-statement",
    InputSerialization={},
    OutputSerialization={},
)
for event in response["Payload"]:
    print(event["Records"]["Payload"].decode("utf-8")

Actual output

temp.py:15: error: "str" has no attribute "decode"; maybe "encode"?  [attr-defined]
temp.py:15: error: Invalid index type "str" for "str"; expected type "Union[SupportsIndex, slice]"  [index]

Expected output

No typing errors thrown.

Additional context

OS: Ubuntu 20.04 on Microsoft Windows Subsystem for Linux (WSL) sys.version: 3.8.10 (default, Mar 13 2023, 10:26:41) \n[GCC 9.4.0] boto3-stubs installation method: pip boto3-stubs version: 1.26.120 bogo3-stubs-s3 version: 1.26.62 boto3 version: 1.18.65

vemel commented 1 year ago

Thank you for the report!

I will try to fix is by Monday.

vemel commented 1 year ago

Aaaand, finally it is Monday. :)

Adding EventStream support was more complex than I expected. Finally, the code is in place, so expect a new release tomorrow.

vemel commented 1 year ago

Published mypy_boto3_s3_package 1.28.3.post2 with the fix included.

Please let me know if it works as it should when you have time.