boto / boto3

AWS SDK for Python
https://aws.amazon.com/sdk-for-python/
Apache License 2.0
9.04k stars 1.87k forks source link

Custom Headers in Presigned URLs Construction #1294

Open kasured opened 7 years ago

kasured commented 7 years ago

Python: 2.7.5 Boto: 2.48.0 Boto3: 1.4.7 Botocore: 1.7.19

Query String Request Authentication Alternative specifies the format of the StringToSign that might be followed to create a signature of the pre-signed url. Namely,

Signature = URL-Encode( Base64( HMAC-SHA1( YourSecretAccessKeyID, UTF-8-Encoding-Of( StringToSign ) ) ) );

StringToSign = HTTP-VERB + "\n" +
    Content-MD5 + "\n" +
    Content-Type + "\n" +
    Expires + "\n" +
    CanonicalizedAmzHeaders +
    CanonicalizedResource;    

The use case is to be able to provide a custom header like Content-MD5 to participate in signature construction.

This is perfectly fine with the boto library

import boto 

## [s3]
## use-sigv4 = True
boto.config.load_from_path('.boto')

conn = boto.connect_s3(profile_name = "_profile_name_", host = "s3.amazonaws.com")

presigned_url = conn.generate_url(
    expires_in=3600,
    method='GET',
    bucket='_bucket_name_',
    key='_key_',
    headers={'Content-MD5': 'c3993a7637ae5d90e70106050ef6926f'}
)

Seems like this same feature cannot be achieved with boto3.

Can this be confirmed and what is the reason behind degrading this feature from the new version of the library? May it be expected to return in the future versions?

kyleknap commented 7 years ago

Would not be opposed to adding support for custom headers. Could you elaborate on why you need to set the Content-MD5 for a GET request? Typically you would not need to set that a header for GET's and this can be seen in the documentation you linked where the StringToSign in the example had no Content-MD5 included (i.e. it was just an empty string).

kasured commented 7 years ago

I would rather we do keep separate flavors of the issue described here. I specifically stressed out the possibility to have the signature be customized in terms of adding additional components that participate in the process.

Content-MD5 being described as "meaningless" in the GET requests does not disallow it to be present. After all, empty value still gives you a hash albeit deterministic.

The use case described just brings up some additional custom contract to the process. If say Alice and Bob decide to include the custom header in the process of signing url requests, than Charlie being not a part of that contract will have an additional step to be able to get a pre-signed url working for him.

There are lots of scenarios that might be exampled here, without stressing out a specific header.

In the meantime, other SDKs like AWS Java SDK and a "hand-coded" boto library does support that feature without violating any strict contract from the AWS Signing Interface.

fifthpostulate commented 6 years ago

I have seen a couple requests for this, and would find it useful myself. Any chance of it happening?

ATGiant commented 6 years ago

Ditto. This would be a great feature.

ayqazi commented 6 years ago

Presigned URLs absolutely require custom metadata to be attached, or else how can one trigger a job off an S3 upload, and keep track of it without attaching a UUID metadatum to it?

dmyur commented 5 years ago

I wish I could add custom headers to my presigned URLs with boto3 😭

pnminh commented 5 years ago

have we got any good news yet? I'm desperately in need with some extra custom headers I want to add.

Dizzzmas commented 4 years ago

Would really like to see those added. Could allow passing custom metadata to the uploaded objects https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingMetadata.html#object-metadata

github-actions[bot] commented 3 years ago

Greetings! It looks like this issue hasn’t been active in longer than one year. We encourage you to check if this is still an issue in the latest release. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or upvote with a reaction on the initial post to prevent automatic closure. If the issue is already closed, please feel free to open a new one.

yanivpaz commented 2 years ago

The issue still exist ....

yanivpaz commented 2 years ago

see also https://github.com/boto/boto3/discussions/3342