boto / boto3

AWS SDK for Python
https://aws.amazon.com/sdk-for-python/
Apache License 2.0
8.99k stars 1.86k forks source link

getting a presigned URL for a key with a leading slash returns a URL whose path starts with a double slash #4181

Closed SpoonMeiser closed 2 months ago

SpoonMeiser commented 3 months ago

Describe the bug

If you have a key with a leading slash, like /my/key.txt, then generating a presigned URL for it will produce a URL that looks like https://blah-blah.s3.amazon.com//my/key.txt?...

It's not clear to me whether the // at the start of the URL path there makes this URL technically invalid or not. However, some other libraries take the view that it is invalid. for example, the popular requests library has recently introduced behaviour that stips the double slash, and this breaks using such presigned URLs: https://github.com/psf/requests/issues/6711

requests is unlikely to be the only library that has issues with such URLs.

Expected Behavior

Generate a URL that doesn't a double slash.

Current Behavior

Produces a URL where the path starts with a //

Reproduction Steps

Adapted from the reproducer in https://github.com/psf/requests/issues/6711 to not be requests specific:

from urllib.parse import urlparse
import boto3

s3 = boto3.client('s3')
bucket = "bucket"
key = "/key_with_leading_slash.txt"
presigned_url = s3.generate_presigned_url("get_object", Params={'Bucket': bucket, 'Key': key})
parsed_url = urlparse(presigned_url)

assert not parsed_url.path.startswith("//")

Possible Solution

I think the Java SDK escapes slashes in this specific case?

Additional Information/Context

No response

SDK version used

1.34.118

Environment details (OS name and version, etc.)

linux ubuntu

tim-finnigan commented 2 months ago

Thanks for reaching out. The team does not plan to make changes here because the presigned url generation is behaving as expected. We recommend following the S3 documentation on naming object keys. You should be able to encode the first slash for requests, here is related documentation: https://requests.readthedocs.io/en/latest/user/advanced/#prepared-requests.

github-actions[bot] commented 2 months ago

This issue is now closed. Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one.