boto / boto3

AWS SDK for Python
https://aws.amazon.com/sdk-for-python/
Apache License 2.0
8.95k stars 1.86k forks source link

S3 multi-part part number type documentation #3501

Open mdavis-xyz opened 1 year ago

mdavis-xyz commented 1 year ago

Describe the issue

The documentation for S3 multipart upload Part says:

multipart_upload_part = multipart_upload.Part('part_number') Parameters part_number (string) -- The Part's part_number identifier. This must be set.

If I use a string (Part('1')) I get an error. If I use an int Part(1) I don't get an error. So the library behavior does not match the documentation.

Steps to reproduce

MWE

import boto3

bucket = 'telstra-energy-test-temp'
key = 'test-multi-part-upload'

s3 = boto3.resource('s3')
obj = s3.Object(bucket, key)
mpu = obj.initiate_multipart_upload()

parts = []
part = mpu.Part('1')
data = b"x" * (8 * 2 ** 20)
resp = part.upload(Body=data)

print("Completing")
mpu.complete(MultipartUpload={
    'Parts': [
        {
            'ETag': resp['ETag'],
            'PartNumber': 1
        }
    ]
})

Run this script.

Expected behavior

Script completes successfully using Part('1'). Multipart upload completed.

Actual behavior

botocore.exceptions.ParamValidationError: Parameter validation failed:
Invalid type for parameter PartNumber, value: 1, type: <class 'str'>, valid types: <class 'int'>

If I change Part('1') to Part(1) the exception is no longer thrown.

Links

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.MultipartUpload.Part

tim-finnigan commented 1 year ago

Thanks @mdavis-xyz for reporting this issue.

jana-personal commented 1 year ago

@tim-finnigan I would like to fix this issue, So we have to fix the document or code?

mdavis-xyz commented 1 year ago

I think we can just change the documentation. An int is more intuitive than a string.

hannes-ucsc commented 1 year ago

think we can just change the documentation.

And the stubs, right?

image
ztravis commented 7 months ago

Wanted to raise this again - we use autogenerated type hints (https://github.com/vemel/mypy_boto3_builder) which make it very easy to introduce this runtime bug. I'm be happy to help fix this although I have to admit I don't know where the particular change would be (I assume the docs are autogenerated somehow...).

vemel commented 1 month ago

@ztravis @hannes-ucsc

The issue was addressed in the latest boto3-stubs. Short explanation: https://github.com/youtype/mypy_boto3_builder/releases/tag/7.26.0

If the issue is related to auto-generated type annotations for boto3, please report them here: https://github.com/youtype/mypy_boto3_builder/issues

hannes-ucsc commented 3 weeks ago

Thank you! I can confirm that release 1.35.1 of boto3-stubs fixes the issue for me.