Support `STREAMING-AWS4-HMAC-SHA256-PAYLOAD`

[vlovich]

Describe the feature

Hi, I'd like to upload a signed body payload without having the actual body upfront (I do know the length up-front but that's it). In theory I should be using STREAMING-AWS4-HMAC-SHA256-PAYLOAD but I can't figure out how to use the SDK to do that. Is this feature not implemented? If not, is there a simple middleware layer I can install to do this?

Use Case

I have some middleware server code responsible for uploading large files to S3. I don't want to have to buffer the file in memory / disk because I'm handling a lot of concurrent requests. I do want to have some confidence that Amazon is validating the signature of the upload cryptographically. I'd like to send the data using STREAMING-AWS4-HMAC-SHA256-PAYLOAD but I don't know how to get the library to do that.

Proposed Solution

No response

Other Information

No response

Acknowledgements

• [ ] I may be able to implement this feature request
• [ ] This feature might incur a breaking change

SDK version used

3.145.0

Environment details (OS name and version, etc.)

Arch Linux

[RanVaknin]

Hi @vlovich ,

Thanks for opening this issue. I guess I'm not sure how you want to calculate the signature if you do not know the body? The signing process takes into account the actual body so I don't see a way to do what you are asking.

Am I misunderstanding your request?

Thansk, Ran~

[vlovich]

Have you seen streaming sigv4? https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html

The Authorization signature just says "I'm going to stream the body in a streaming sigv4 fashion" (STREAMING-AWS4-HMAC-SHA256-PAYLOAD). The body is sent in chunks that are themselves cryptographically signed.

For example, rather than including the sha256 of the body upfront (& thus needing to sha256 the entire file upfront), I can still get a signed upload for a 5 GiB file while only needing to have a small in-memory buffer for computing the signature (smaller latency too although in my use-case I don't have the full body upfront and have limited working RAM compared with the size of files I'm processing).

This is done, for example, by buffering 256 kib of body, compute the signature, send the chunk. The signature for each chunk is ratcheted (i.e. signature for chunk 2 is signature for chunk 1 + hash of empty string + hash of payload in chunk 2). That way you don't need to precompute the hash upfront but you still get a cryptographically signed upload.

[eyiu-ca]

Is there any update on this feature request?