AwsV4HttpSigner does not contain all features from Aws4Signer

MatejNedic commented 4 months ago

Describe the feature

Introduce presign option like in Aws4Signer since it is useful when you don't want to issue it with http request call.

AwsV4HttpSigner does not support this option.

Use Case

Aws4Signer had a feature to presign requests which was useful when you wanted to use SIGV4 and use for example Spring Boots WebClient to set it in header or body.

Check following docs here and search for presign method.

Proposed Solution

No response

Other Information

No response

Acknowledgements

[X] I may be able to implement this feature request
[ ] This feature might incur a breaking change

AWS Java SDK version used

2.26.0

JDK version used

17 Corretto

Operating System and version

Mac OS

debora-ito commented 3 months ago

@MatejNedic

The equivalent of presign in AwsV4HttpSigner is to set property AwsV4FamilyHttpSigner.AUTH_LOCATION to QUERY_STRING.

Let us know if this is not what you're looking for.

MatejNedic commented 3 months ago

Hey @debora-ito , thanks for quick response!

Please correct me if I am wrong, but this is how I got it.

If I have service to service communication which can use AWS sigv4 I would have to unpack signed request to get signature right (lets say I am not using Apache HTTP client where you can just pass request)?

First example and use case here. I have machine to machine communication and I am using STS. I am only interested in signature since I want to use it in header of my HTTPS request.

Would it make sense to have presign option since this gives me only signature which I am only interested at? I really think this would simplify use of sigv4 in machine to machine communication. I have feeling current implementation is S3 and AWS service heavy (intended only for that use) making it hard to use custom header names for example.

Second example I see in theory you could cache the signature for very short time spans and this lowering https calls you are making to AWS as well. (Although not sure how secure this really is).

As third example I see more power and easier of use when integrating this in Spring WebClient, since they do serialisation/deserialisation for you meaning we are moving from HTTP request to Object -> Json, Header -> Map<> and this makes integration dirty to unwrap SignedRequest to get signature.

Edit: Forgot to add use case four when using gRPC communication I might want to have it in metadata part.

KroArtem commented 1 month ago

I think we have stumbled upon the same issue after updating from 2.20.57 to 2.26.31. Our tests with multipart upload started failing, after adding debug output I saw a difference between those two versions:

before:

DEBUG s.a.a.c.i.ExecutionInterceptorChain - Creating an interceptor chain that will apply interceptors in the following order: [software.amazon.awssdk.services.s3.endpoints.internal.S3ResolveEndpointInterceptor@4433e91b, software.amazon.awssdk.services.s3.endpoints.internal.S3EndpointAuthSchemeInterceptor@778aff46, software.amazon.awssdk.services.s3.endpoints.internal.S3RequestSetEndpointInterceptor@336a704e]
DEBUG s.a.a.c.i.ExecutionInterceptorChain - Interceptor 'software.amazon.awssdk.services.s3.endpoints.internal.S3EndpointAuthSchemeInterceptor@778aff46' modified the message with its modifyRequest method.
DEBUG s.a.a.c.i.ExecutionInterceptorChain - Interceptor 'software.amazon.awssdk.services.s3.endpoints.internal.S3RequestSetEndpointInterceptor@336a704e' modified the message with its modifyHttpRequest method.
DEBUG s.a.awssdk.auth.signer.Aws4Signer - AWS4 Canonical Request: POST
/space-local/9b10gffuqc
X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=app-access-key%2F20241004%2Feu-west-1%2Fs3%2Faws4_request&X-Amz-Date=20241004T133706Z&X-Amz-Expires=60&X-Amz-SignedHeaders=content-length%3Bcontent-type%3Bhost&uploadId=ZmU1NzM2NWItMTBjYy00ZTkzLWIwM2EtMzBlNzJhNzI0MWY2Ljk1YWU0ODA4LWJiNmUtNDJjOS04Nzc3LTQ1OTFlNmQzMWU1MHgxNzI4MDQ5MDI2MDU5ODUyMzM2
content-length:233
content-type:application/xml
host:localhost:29005

content-length;content-type;host
UNSIGNED-PAYLOAD
2024-10-05 01:37:06.133 [ui dispatch thread @runAsyncAndBlock#612] DEBUG s.a.awssdk.auth.signer.Aws4Signer - AWS4 String to sign: AWS4-HMAC-SHA256
20241004T133706Z
20241004/eu-west-1/s3/aws4_request
dd8f9dc56b062d66dcad213bc923027f4b882e69eac553a8479e6ee383902442
DEBUG s.a.a.c.i.ExecutionInterceptorChain - Creating an interceptor chain that will apply interceptors in the following order: [software.amazon.awssdk.services.s3.endpoints.internal.S3ResolveEndpointInterceptor@4433e91b, software.amazon.awssdk.services.s3.endpoints.internal.S3EndpointAuthSchemeInterceptor@778aff46, software.amazon.awssdk.services.s3.endpoints.internal.S3RequestSetEndpointInterceptor@336a704e]
DEBUG s.a.a.c.i.ExecutionInterceptorChain - Interceptor 'software.amazon.awssdk.services.s3.endpoints.internal.S3EndpointAuthSchemeInterceptor@778aff46' modified the message with its modifyRequest method.
DEBUG s.a.a.c.i.ExecutionInterceptorChain - Interceptor 'software.amazon.awssdk.services.s3.endpoints.internal.S3RequestSetEndpointInterceptor@336a704e' modified the message with its modifyHttpRequest method.
DEBUG s.a.awssdk.auth.signer.Aws4Signer - AWS4 Canonical Request: POST
/space-local/9b10gffuqc
X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=app-access-key%2F20241004%2Feu-west-1%2Fs3%2Faws4_request&X-Amz-Date=20241004T133706Z&X-Amz-Expires=60&X-Amz-SignedHeaders=content-length%3Bcontent-type%3Bhost&uploadId=ZmU1NzM2NWItMTBjYy00ZTkzLWIwM2EtMzBlNzJhNzI0MWY2Ljk1YWU0ODA4LWJiNmUtNDJjOS04Nzc3LTQ1OTFlNmQzMWU1MHgxNzI4MDQ5MDI2MDU5ODUyMzM2
content-length:233
content-type:application/xml
host:localhost:29005

content-length;content-type;host
UNSIGNED-PAYLOAD
DEBUG s.a.awssdk.auth.signer.Aws4Signer - AWS4 String to sign: AWS4-HMAC-SHA256
20241004T133706Z
20241004/eu-west-1/s3/aws4_request
dd8f9dc56b062d66dcad213bc923027f4b882e69eac553a8479e6ee383902442

After:

DEBUG s.a.a.c.i.ExecutionInterceptorChain - Creating an interceptor chain that will apply interceptors in the following order: [software.amazon.awssdk.services.s3.auth.scheme.internal.S3AuthSchemeInterceptor@46d02058, software.amazon.awssdk.services.s3.endpoints.internal.S3ResolveEndpointInterceptor@58d7b444, software.amazon.awssdk.services.s3.endpoints.internal.S3RequestSetEndpointInterceptor@7a2c9f19]
DEBUG s.a.a.c.i.ExecutionInterceptorChain - Interceptor 'software.amazon.awssdk.services.s3.endpoints.internal.S3RequestSetEndpointInterceptor@7a2c9f19' modified the message with its modifyHttpRequest method.
DEBUG s.a.a.h.a.a.i.s.DefaultAwsV4HttpSigner - Payload signing was disabled for an HTTP request with a payload. Signing will be enabled. Use HTTPS for unsigned payloads.
DEBUG s.a.a.h.a.a.i.s.DefaultAwsV4HttpSigner - Payload signing was disabled for an HTTP request with a payload. Signing will be enabled. Use HTTPS for unsigned payloads.
DEBUG s.a.a.h.a.a.i.s.DefaultV4RequestSigner - AWS4 Canonical Request: POST
/space-local/cuqeph2sz4
X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=app-access-key%2F20241004%2Feu-west-1%2Fs3%2Faws4_request&X-Amz-Date=20241004T120411Z&X-Amz-Expires=60&X-Amz-SignedHeaders=content-length%3Bcontent-type%3Bhost&uploadId=NTQ3YjRkNjgtMTRjNC00ZTczLTk4MDMtMTU2MTJkOTFmZDk5LmEyYzRkZDM0LWNjNGMtNDNhYi1hYTMzLWI5NTNiZWY4NGI1NngxNzI4MDQzNDUwOTYzMTE4MDA5
content-length:233
content-type:application/xml
host:localhost:29005

content-length;content-type;host
ebe22c14f554aaef0a92395502f06e7d0f88ceb58b1e2e81f4043cf77f46093f
DEBUG s.a.a.h.a.a.i.s.DefaultV4RequestSigner - AWS4 Canonical Request Hash: 8d53911701b20634f7b0087e9818ea5c5ab02c56766cb480545d5866d6fec0b8
DEBUG s.a.a.h.a.a.i.s.DefaultV4RequestSigner - AWS4 String to sign: AWS4-HMAC-SHA256
20241004T120411Z
20241004/eu-west-1/s3/aws4_request
8d53911701b20634f7b0087e9818ea5c5ab02c56766cb480545d5866d6fec0b8
DEBUG s.a.a.c.i.ExecutionInterceptorChain - Creating an interceptor chain that will apply interceptors in the following order: [software.amazon.awssdk.services.s3.auth.scheme.internal.S3AuthSchemeInterceptor@46d02058, software.amazon.awssdk.services.s3.endpoints.internal.S3ResolveEndpointInterceptor@58d7b444, software.amazon.awssdk.services.s3.endpoints.internal.S3RequestSetEndpointInterceptor@7a2c9f19]
DEBUG s.a.a.c.i.ExecutionInterceptorChain - Interceptor 'software.amazon.awssdk.services.s3.endpoints.internal.S3RequestSetEndpointInterceptor@7a2c9f19' modified the message with its modifyHttpRequest method.
DEBUG s.a.a.h.a.a.i.s.DefaultAwsV4HttpSigner - Payload signing was disabled for an HTTP request with a payload. Signing will be enabled. Use HTTPS for unsigned payloads.
DEBUG s.a.a.h.a.a.i.s.DefaultAwsV4HttpSigner - Payload signing was disabled for an HTTP request with a payload. Signing will be enabled. Use HTTPS for unsigned payloads.
DEBUG s.a.a.h.a.a.i.s.DefaultV4RequestSigner - AWS4 Canonical Request: POST
/space-local/cuqeph2sz4
X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=app-access-key%2F20241004%2Feu-west-1%2Fs3%2Faws4_request&X-Amz-Date=20241004T120411Z&X-Amz-Expires=60&X-Amz-SignedHeaders=content-length%3Bcontent-type%3Bhost&uploadId=NTQ3YjRkNjgtMTRjNC00ZTczLTk4MDMtMTU2MTJkOTFmZDk5LmEyYzRkZDM0LWNjNGMtNDNhYi1hYTMzLWI5NTNiZWY4NGI1NngxNzI4MDQzNDUwOTYzMTE4MDA5
content-length:233
content-type:application/xml
host:localhost:29005

content-length;content-type;host
ebe22c14f554aaef0a92395502f06e7d0f88ceb58b1e2e81f4043cf77f46093f
DEBUG s.a.a.h.a.a.i.s.DefaultV4RequestSigner - AWS4 Canonical Request Hash: 8d53911701b20634f7b0087e9818ea5c5ab02c56766cb480545d5866d6fec0b8
DEBUG s.a.a.h.a.a.i.s.DefaultV4RequestSigner - AWS4 String to sign: AWS4-HMAC-SHA256
20241004T120411Z
20241004/eu-west-1/s3/aws4_request
8d53911701b20634f7b0087e9818ea5c5ab02c56766cb480545d5866d6fec0b8

So my guess is that HTTP request was signed, thus we had no UNSIGNED-PAYLOAD in the output and signature has changed.

@debora-ito , any chances this usecase could be taken into consideration? Thanks in advance.

KroArtem commented 1 month ago

@debora-ito , sorry for bothering you again, just wanted to know whether my assumptions were correct or should I dig further?

KroArtem commented 2 weeks ago

I think https://github.com/aws/aws-sdk-java-v2/issues/4196 and https://github.com/aws/aws-sdk-java-v2/issues/5683 are pretty much related to this one.

aws / aws-sdk-java-v2