Move to Signature Version 4 Signing

[kantp]

To use the regions China (Beijing) or EU (Frankfurt), we need to sign requests with Signature Version 4, since they do not support Version 2. Also, AWS strongly recommends to use V4 over V2.

There's already the package https://github.com/iconnect/aws-sign4 that allows signing a request using V4, and the https://github.com/iconnect/aws-elastic-transcoder package that uses it. I think it would be beneficial to use that for signing requests for the other services as well. However, to use V4 signatures in the other services, that functionality would have to be moved to the core library in order to avoid cyclic dependencies.

So my question is: was there a specific reason for implementing aws-sign4 as a separate package? Would it be ok for the author, @cdornan, to have this moved into the aws package?

If there's no reason against this (license restrictions maybe?), I could volunteer to do the administrative work of moving Sign4.hs to the main repository, and start on switching to V4 for the S3 commands.

[aristidb]

No fundamental reason, but please ask @cdornan if he's OK with this.

Philipp notifications@github.com schrieb am Mo., 22. Juni 2015 12:40:

To use the regions China (Beijing) or EU (Frankfurt), we need to sign requests with Signature Version 4, since they do not support Version 2. Also, AWS strongly recommends to use V4 over V2.

There's already the package https://github.com/iconnect/aws-sign4 that allows signing a request using V4, and the https://github.com/iconnect/aws-elastic-transcoder package that uses it. I think it would be beneficial to use that for signing requests for the other services as well. However, to use V4 signatures in the other services, that functionality would have to be moved to the core library in order to avoid cyclic dependencies.

So my question is: was there a specific reason for implementing aws-sign4 as a separate package? Would it be ok for the author, @cdornan https://github.com/cdornan, to have this moved into the aws package?

If there's no reason against this (license restrictions maybe?), I could volunteer to do the administrative work of moving Sign4.hs to the main repository, and start on switching to V4 for the S3 commands.

— Reply to this email directly or view it on GitHub https://github.com/aristidb/aws/issues/167.

[sigrlami]

Any updates on this?

[aristidb]

@sigrlami No, I haven't got a message from @cdornan - but if this would be useful for you, please go ahead and make a pull request.

[sigrlami]

@aristidb yes, that will be very useful. I'll work this out and prepare PR. Thanks

[angerman]

@sigrlami did you get anywhere with the PR for this?

[agrafix]

@angerman I've tried out your commit https://github.com/angerman/aws/commit/cbdb8d4704db06880629a38aea14e29386194f65 but I receive a status 403 code indicating "The request signature we calculated does not match the signature you provided. Check your key and signing method.". I've tried to upload using S3.multipartUploadSink

Using S3.putObject and manually setting poContentSha256 via cryptohash-conduit (https://hackage.haskell.org/package/cryptohash-conduit-0.1.1/docs/Crypto-Hash-Conduit.html) results in a status 400: "x-amz-content-sha256 must be UNSIGNED-PAYLOAD, STREAMING-AWS4-HMAC-SHA256-PAYLOAD, or a valid sha256 value."

[angerman]

@agrafix, thanks for reporting. I'm sorry for causing any premature noise by referencing this issue in one of the commits. I hope to be able to fix this for good over the weekend. The sha256 record fields were a mistake.

@agrafix can you provide a short sample of how you are using it?

[angerman]

@agrafix please try again with #199. S3.putObject should work. multi-part seems to be a bit more involved in signature calculations.

If you prefer, you can set the content sha256 (by default the payload is unsigned) using some construction like this:

let put = (S3.putObject bucket objectName (RequestBodyBS payload)) { poContentSHA256 = Just (hash payload) }

using a payload :: ByteString; hash from Crypto.Hash (crytpohash) and RequestBodyBS from Network.HTTP.Client.

[agrafix]

@angerman great, putObject works now! :-)

[joeyh]

+1 just had a user file a bug on git-annex not working for Beijing. Am surprised nobody complained about Frankfurt yet if it needs this.

[aristidb]

@angerman What is the status of your pull request? Do you consider it a WIP still?

[angerman]

Even though it pains me to say, I've since replaced aws with amazonka, as it came with support for sig v4 (yes, I was concerned precisely for the use with Frankfurt).

If someone wants to take over from here, please do so!

[maartevh]

I try to use S3 with Frankfurt. But it failed. Is there a chance to fixed this?

[aristidb]

@maartevh There is a partial pull request at #199 - but it's fairly complex and hasn't been completed yet. It was written by @angerman who does not use the aws package anymore.

[omnibs]

199 was superseded by https://github.com/aristidb/aws/pull/241 which merged back in 2017. Should this issue be closed or is there any remaining work for V4?