boto / boto3

AWS SDK for Python
https://aws.amazon.com/sdk-for-python/
Apache License 2.0
8.99k stars 1.86k forks source link

generate_presigned_url does not acknowledge range in get_object without specifying signature_version. #2417

Closed pulllow closed 2 years ago

pulllow commented 4 years ago

If you create a client without specifying the signature version in the config it will not honor the range set in the get _object.

boto3 1.13.2 python 3.8

Steps to reproduce

create client without signature version

s3 = boto3.client('s3') url = s3.generate_presigned_url( 'get_object', Params={ 'Key': , 'Bucket': , 'Range' : 'bytes=0-20' }, ExpiresIn=, HttpMethod='GET', )

different_range_headers = {'range': 'bytes=21-39'}

does not acknowledge range in signature and will retrieve bytes

resp = requests.get(url, headers=different_range_headers)

create client with signature version

s3 = boto3.client('s3', config=Config(signature_version='s3v4')) url = s3.generate_presigned_url( 'get_object', Params={ 'Key': , 'Bucket': , 'Range' : 'bytes=0-20' }, ExpiresIn=, HttpMethod='GET', )

different_range_headers = {'range': 'bytes=21-39'}

this will fail 403 as expected

resp = requests.get(url, headers=different_range_headers)

swetashre commented 4 years ago

@pulllow - Thank you for your post. This is the way signature version v2 works with header . For version s3v2 the signature does not factor in the header so it will work even if you don't send it. This is why v2 signature are deprecated and being phased out in favor of V4. SigV4 guarantees the header is present and is a particular value as part of the signature but V2 does not. So you will get error in the second part of code if you specify different header value or omit the header which explains the behavior you are seeing.

pulllow commented 4 years ago

@swetashre - Thanks for looking. If I am reading the documentation correctly s3v4 is the default and if I do not specify a version in the config then the first scenarios above should produce the same result as the second. ... signature_version: The AWS signature version to use when signing requests. When necessary, Boto automatically switches the signature version to an appropriate value. The following values are recognized.

s3v4 (Default) Signature Version 4 s3 (Deprecated) Signature Version 2 ... https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html

swetashre commented 4 years ago

@pulllow - We still use sigv2 for presigned urls unless it has explicitly configured to use sigv4. We should update our documentation. Marking this as documentation update.

github-actions[bot] commented 2 years ago

Greetings! It looks like this issue hasn’t been active in longer than one year. We encourage you to check if this is still an issue in the latest release. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or upvote with a reaction on the initial post to prevent automatic closure. If the issue is already closed, please feel free to open a new one.

mtvx commented 2 months ago

@swetashre So what is the status here - does it default to s3v4 these days also for presigned URLs?

This ticket is closed, but the docs are still a bit vague about it. I get the impression that V4 is used also for presigned URLs these days, but is that right?

Especially:

If you’re using a presigned URL with an expiry of greater than 7 days, you should specify Signature Version 2.

What does should mean? Must or must not?