AmazonApiGatewayManagementApiClient.PostToConnectionAsync wrong signature

[usix79]

Describe the bug

AWSSDK.ApiGatewayManagementApi Version="3.7.300.19"

AmazonApiGatewayManagementApiClient.PostToConnectionAsync it is not capable to properly sign requests to ServiceURL like

https://EDITED.execute-api.eu-central-1.amazonaws.com/$default

Produced canonical resource path:

 /%2524default/%2540connections/P-S_9e-7liACH9A%253D

but expected:

/%24default/%2540connections/P-S_9e-7liACH9A%253D

Expected Behavior

Calling AmazonApiGatewayManagementApiClient.PostToConnectionAsync should not throw a signature validation exception assuming that the correct permissions are in place.

Current Behavior

Exception in AmazonApiGatewayManagementApiClient.PostToConnectionAsync:

AmazonApiGatewayManagementApiException making request PostToConnectionRequest to https://EDITED.execute-api.eu-central-1.amazonaws.com/$default. Attempt 1. --> Amazon.ApiGatewayManagementApi.AmazonApiGatewayManagementApiException: The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.

The Canonical String for this request should have been
'POST
/%24default/%2540connections/P-S_9e-7liACH9A%253D

content-length:1367
content-type:binary/octet-stream
host:EDITED.execute-api.eu-central-1.amazonaws.com
user-agent:aws-sdk-dotnet-coreclr/3.7.300.19 ua/2.0 os/linux#4.14.255.327 md/ARCH#X64 lang/.NET_Core#6.0.22 exec-env/AWS_Lambda_dotnet6 md/aws-sdk-dotnet-core#3.7.300.19 api/ApiGatewayManagementApi#3.7.300.19 cfg/retry-mode#legacy md/ClientAsync
x-amz-api-version:2018-11-29
x-amz-content-sha256:1402029c938875639856ff94f4ee60faaeb80dfab44ed68fd780f7e91615efde
x-amz-date:20231215T072312Z
x-amz-security-token:EDITED

content-length;content-type;host;user-agent;x-amz-api-version;x-amz-content-sha256;x-amz-date;x-amz-security-token
1402029c938875639856ff94f4ee60faaeb80dfab44ed68fd780f7e91615efde'

The String-to-Sign should have been
'AWS4-HMAC-SHA256
20231215T072312Z
20231215/eu-central-1/execute-api/aws4_request
740005dbd3031869909edd3d0100e351e0eb12562d09b2f903309c82e3edbf78'

 ---> Amazon.Runtime.Internal.HttpErrorResponseException: Exception of type 'Amazon.Runtime.Internal.HttpErrorResponseException' was thrown.
   at Amazon.Runtime.HttpWebRequestMessage.ProcessHttpResponseMessage(HttpResponseMessage responseMessage)
   at Amazon.Runtime.HttpWebRequestMessage.GetResponseAsync(CancellationToken cancellationToken)
   at Amazon.Runtime.Internal.HttpHandler`1.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.Unmarshaller.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
   --- End of inner exception stack trace ---
   at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleExceptionStream(IRequestContext requestContext, IWebResponseData httpErrorResponse, HttpErrorResponseException exception, Stream responseStream)
   at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleExceptionAsync(IExecutionContext executionContext, HttpErrorResponseException exception)
   at Amazon.Runtime.Internal.ExceptionHandler`1.HandleAsync(IExecutionContext executionContext, Exception exception)
   at Amazon.Runtime.Internal.ErrorHandler.ProcessExceptionAsync(IExecutionContext executionContext, Exception exception)
   at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.Signer.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CredentialsRetriever.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)

And this is the log entry that precedes the API call:

Double encoded /$default/@connections/{connectionId} with endpoint https://EDITED.execute-api.eu-central-1.amazonaws.com/$default for canonicalization: /%2524default/%2540connections/P-S_9e-7liACH9A%253D

Reproduction Steps

Install any version of AWSSDK.APIGatewayManagementApi 3.7.300.19

var cfg = new AmazonApiGatewayManagementApiConfig { ServiceURL = "https://EDITED.execute-api.eu-central-1.amazonaws.com/$default" }
var client = new AmazonApiGatewayManagementApiClient(credentials, cfg);
await client.PostToConnectionAsync(new PostToConnectionRequest
{
    ConnectionId = "EDITED",
    Data = new System.IO.MemoryStream()
});

Try to post anything to the websocket connection and you will get a signature validation error.

Possible Solution

As workaround - use earlier version of SDK. f.e. "AWSSDK.ApiGatewayManagementApi" Version="3.7.200.26" works good.

Additional Information/Context

No response

AWS .NET SDK and/or Package version used

"AWSSDK.ApiGatewayManagementApi" Version="3.7.300.19"

Targeted .NET Platform

.NET 6

Operating System and version

Amazon Linux 2

[ashishdhingra]

@usix79 Good afternoon. I'm unsure if you need to specify $default route as part of ServiceURL. Per Working with stages for HTTP APIs, $default route is served from the base of your API's URL - for example, https://{api_id}.execute-api.{region}.amazonaws.com/. Use @connections commands in your backend service also doesn't demonstrate use of $default for a route. Kindly note that $default is a route, not a stage. The examples I have seen so far construct endpoint based on stage (perhaps using the $default route).

Could you please share the following:

• Result when you invoke PostToConnectionAsync() with endpoint not explicitly specifying $default for a route?
• How is your WebSocket API setup? As per my understanding, one has to define a stage to deploy the API. Have you defined only a single stage? I'm unsure if one could omit a stage in case only one of them is defined.

Thanks, Ashish

[ashishdhingra]

Using the below steps, I was able to get a quick and loose example AWS API Gateway connection working use MOCK endpoints with web sockets (some of the steps in new AWS Console UI could be different).

Reference: https://stackoverflow.com/questions/55594587/setup-a-basic-websocket-mock-in-aws-apigateway

1. First, go to API Gateway, click Create API
2. For Choose the protocol, select WebSocket
3. Give the API a name, and use the example $request.body.message for the route selection expression
4. Click the Create API button
5. Select the new API from the side pane, and click Routes
6. Select the $connect route
7. Under Route overview, select the Mock radio button for endpoint
8. For the Request Template, enter an expression (using 200 for example).
9. For Template Key, enter a key (using 200 for example)
10. For the popup window on passthrough behavior, select No, Use Current Settings for now
11. Click on the Template Key 200, and enter a template: {"statusCode" : 200} and then click the Save button.
12. Select Route Overview to go back to the $connect overview page and then select the Add Integration Response button
13. Create a response key of $default, which will reference the $default key under Routes connect integration response
14. For the $disconnect route, please repeat steps 7 - 13 for this specific route.
15. Once done setting up the $disconnect route, please select the $default route under the Routes pane.
16. Create an Integration Request the same way that was done for $connect and $disconnect (steps 7 - 11)
17. For Integration Response under $default, we will be setting the route response up slightly different than $connect and $disconnect as those 2 routes' responses actually reference the $default response within their own respective Integration Responses.
18. Create a response key of $default
19. Enter a Template Selection Expression (using 200 for this example)
20. Then create a Template Key (using 200 for this example)
21. Click on the Template Key 200, and enter a template: {"statusCode" : 200, "connectionId" : "$context.connectionId"} and then click the Save button.
22. {"connectionId" : "$context.connectionId"} was added to the response payload to show how variables can be used within the response to represent an actual value of the connection.
23. Also, in API Gateway > APIs > Settings, specify CloudWatch log role ARN for logging to CloudWatch.
24. Create a Test stage with CloudWatch logs set to Error and info logs.
25. Finally, under the Actions dropdown, select Deploy API, and enter a deployment stage name. This should generate a WebSocket URL that can now be connected to e.g. (wss://<<api-id>>.execute-api.us-east-2.amazonaws.com/Test; get the api-id from AWS console)
26. Using wscat via cli, I am now able to connect to my WebSocket URL (see link for installing wscat): wscat -c wss://<<api-id>>.execute-api.us-east-1.amazonaws.com/Test

Once the connected using wscat, open CloudWatch logs and get the connectionId (as we enabled it in step 21 above). Now use below code:

using Amazon.ApiGatewayManagementApi;
using Amazon.ApiGatewayManagementApi.Model;
using System.Net;

Amazon.AWSConfigs.LoggingConfig.LogResponses = Amazon.ResponseLoggingOption.Always;
Amazon.AWSConfigs.LoggingConfig.LogTo = Amazon.LoggingOptions.SystemDiagnostics;
Amazon.AWSConfigs.AddTraceListener("Amazon", new System.Diagnostics.ConsoleTraceListener());

string apiId = "<<api-id>>";
string stageName = "<<stage-name>>"; // Test for this scenario
string connectionId = "<<connection-id>>"; // As obtained from logs.
var cfg = new AmazonApiGatewayManagementApiConfig { ServiceURL = $"https://{apiId}.execute-api.us-east-2.amazonaws.com/{stageName}" };
var client = new AmazonApiGatewayManagementApiClient(cfg);
await client.PostToConnectionAsync(new PostToConnectionRequest
{
    ConnectionId = connectionId,
    Data = new System.IO.MemoryStream()
});

This works successfully giving below output:

Amazon Information: 0 : The environment variable AWS_ENABLE_ENDPOINT_DISCOVERY was not set with a value.
Amazon Information: 1 : The environment variable AWS_MAX_ATTEMPTS was not set with a value.
Amazon Information: 2 : The environment variable AWS_RETRY_MODE was not set with a value.
Amazon Information: 3 : The environment variable AWS_EC2_METADATA_SERVICE_ENDPOINT was not set with a value.
Amazon Information: 4 : The environment variable AWS_EC2_METADATA_SERVICE_ENDPOINT_MODE was not set with a value.
Amazon Information: 5 : The environment variable AWS_USE_DUALSTACK_ENDPOINT was not set with a value.
Amazon Information: 6 : The environment variable AWS_USE_FIPS_ENDPOINT was not set with a value.
Amazon Information: 7 : The environment variable AWS_IGNORE_CONFIGURED_ENDPOINT_URLS was not set with a value.
Amazon Information: 8 : The environment variable AWS_DISABLE_REQUEST_COMPRESSION was not set with a value.
Amazon Information: 9 : The environment variable AWS_REQUEST_MIN_COMPRESSION_SIZE_BYTES was not set with a value.
Amazon Information: 10 : The environment variable AWS_SDK_UA_APP_ID was not set with a value.
Amazon Information: 0 : There is no defaults_mode set in the profile named 'default' in store Amazon.Runtime.CredentialManagement.CredentialProfileStoreChain
Amazon Information: 1 : There is no endpoint_discovery_enabled set in the profile named 'default' in store Amazon.Runtime.CredentialManagement.CredentialProfileStoreChain
Amazon Information: 2 : There is no retry_mode set in the profile named 'default' in store Amazon.Runtime.CredentialManagement.CredentialProfileStoreChain
Amazon Information: 3 : There is no max_attempts set in the profile named 'default' in store Amazon.Runtime.CredentialManagement.CredentialProfileStoreChain
Amazon Information: 4 : There is no ec2_metadata_service_endpoint set in the profile named 'default' in store Amazon.Runtime.CredentialManagement.CredentialProfileStoreChain
Amazon Information: 5 : There is no ec2_metadata_service_endpoint_mode set in the profile named 'default' in store Amazon.Runtime.CredentialManagement.CredentialProfileStoreChain
Amazon Information: 6 : There is no use_dualstack_endpoint set in the profile named 'default' in store Amazon.Runtime.CredentialManagement.CredentialProfileStoreChain
Amazon Information: 7 : There is no use_fips_endpoint set in the profile named 'default' in store Amazon.Runtime.CredentialManagement.CredentialProfileStoreChain
Amazon Information: 8 : ignore_configured_endpoint_urls found in profile 'default' in store Amazon.Runtime.CredentialManagement.CredentialProfileStoreChain
Amazon Information: 9 : There is no endpoint_url set in the profile named 'default' in store Amazon.Runtime.CredentialManagement.CredentialProfileStoreChain
Amazon Information: 10 : There is no disable_request_compression set in the profile named 'default' in store Amazon.Runtime.CredentialManagement.CredentialProfileStoreChain
Amazon Information: 11 : There is no request_min_compression_size_bytes set in the profile named 'default' in store Amazon.Runtime.CredentialManagement.CredentialProfileStoreChain
Amazon Information: 12 : There is no sdk_ua_app_id set in the profile named 'default' in store Amazon.Runtime.CredentialManagement.CredentialProfileStoreChain
Amazon Information: 0 : Resolved DefaultConfigurationMode for RegionEndpoint [] to [Legacy].
Amazon Information: 0 : Starting a process with the following ProcessInfo: UseShellExecute - False RedirectStandardError - True, RedirectStandardOutput - True, CreateNoWindow - True
Amazon Information: 1 : Process started
Amazon Information: 0 : Process ends with exitcode - 0
Amazon Verbose: 2 : Double encoded /Test/@connections/{connectionId} with endpoint https://<<API-ID>>.execute-api.us-east-2.amazonaws.com/Test for canonicalization: /Test/%2540connections/QNJwacQniYcCGXQ%253D
Amazon Verbose: 0 : Received response (truncated to 1024 bytes): []

Looks like API Gateway allows to create stage named $default for WebSocket API. (However, it does not allow $ character in route key). EDIT: Per https://docs.aws.amazon.com/apigatewayv2/latest/api-reference/apis-apiid-stages-stagename.html, Stage names can contain only alphanumeric characters, hyphens, and underscores, or be $default. AWS Console allows only $default (with $ character in stage name), otherwise it throws error Stage name only allows a-zA-Z0-9_.

• Created a stage named $default
• Deployed API, selecting $default as stage.
• Once API is deployed, used wscat -c wss://<<api-id>>.execute-api.us-east-2.amazonaws.com/$default/. Somehow wscat returns error error: Unexpected server response: 400
• However, running the above C# code does return the error reported by user:

  
  Amazon Error: 0 : An exception of type HttpErrorResponseException was handled in ErrorHandler., Amazon.Runtime.Internal.HttpErrorResponseException: Exception of type 'Amazon.Runtime.Internal.HttpErrorResponseException' was thrown.
  at Amazon.Runtime.HttpWebRequestMessage.ProcessHttpResponseMessage(HttpResponseMessage responseMessage)
  at Amazon.Runtime.HttpWebRequestMessage.GetResponseAsync(CancellationToken cancellationToken)
  at Amazon.Runtime.Internal.HttpHandler`1.InvokeAsync[T](IExecutionContext executionContext)
  at Amazon.Runtime.Internal.Unmarshaller.InvokeAsync[T](IExecutionContext executionContext)
  at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
  Amazon Error: 1 : Received error response: [{"message":"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.\n\nThe Canonical String for this request should have been\n'POST\n/%24default/%2540connections/QNJwacQniYcCGXQ%253D\n\ncontent-length:0\ncontent-type:application/x-www-form-urlencoded; charset=utf-8\nhost:8vnardz1z8.execute-api.us-east-2.amazonaws.com\nuser-agent:aws-sdk-dotnet-coreclr/3.7.300.21 ua/2.0 os/windows#10.0.19045.0 md/ARCH#X64 lang/.NET_Core#6.0.25 md/aws-sdk-dotnet-core#3.7.300.21 api/ApiGatewayManagementApi#3.7.300.21 cfg/retry-mode#legacy md/ClientAsync\nx-amz-api-version:2018-11-29\nx-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\nx-amz-date:20231219T195004Z\nx-amz-security-token:<<REDACTED>>\n\ncontent-length;content-type;host;user-agent;x-amz-api-version;x-amz-content-sha256;x-amz-date;x-amz-security-token\ne3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'\n\nThe String-to-Sign should have been\n'AWS4-HMAC-SHA256\n20231219T195004Z\n20231219/us-east-2/execute-api/aws4_request\n37208530f452c3e9254cbd755f807cb6b88f3d9b24fc803e64c9373deeb8227c'\n"}], Amazon.ApiGatewayManagementApi.AmazonApiGatewayManagementApiException: The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.

The Canonical String for this request should have been 'POST /%24default/%2540connections/QNJwacQniYcCGXQ%253D

content-length:0 content-type:application/x-www-form-urlencoded; charset=utf-8 host:8vnardz1z8.execute-api.us-east-2.amazonaws.com user-agent:aws-sdk-dotnet-coreclr/3.7.300.21 ua/2.0 os/windows#10.0.19045.0 md/ARCH#X64 lang/.NET_Core#6.0.25 md/aws-sdk-dotnet-core#3.7.300.21 api/ApiGatewayManagementApi#3.7.300.21 cfg/retry-mode#legacy md/ClientAsync x-amz-api-version:2018-11-29 x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 x-amz-date:20231219T195004Z x-amz-security-token:<>

content-length;content-type;host;user-agent;x-amz-api-version;x-amz-content-sha256;x-amz-date;x-amz-security-token e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'

The String-to-Sign should have been 'AWS4-HMAC-SHA256 20231219T195004Z 20231219/us-east-2/execute-api/aws4_request 37208530f452c3e9254cbd755f807cb6b88f3d9b24fc803e64c9373deeb8227c'

---> Amazon.Runtime.Internal.HttpErrorResponseException: Exception of type 'Amazon.Runtime.Internal.HttpErrorResponseException' was thrown. at Amazon.Runtime.HttpWebRequestMessage.ProcessHttpResponseMessage(HttpResponseMessage responseMessage) at Amazon.Runtime.HttpWebRequestMessage.GetResponseAsync(CancellationToken cancellationToken) at Amazon.Runtime.Internal.HttpHandler`1.InvokeAsync[T](IExecutionContext executionContext) at Amazon.Runtime.Internal.Unmarshaller.InvokeAsync[T](IExecutionContext executionContext) at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext) --- End of inner exception stack trace --- Amazon Error: 2 : AmazonApiGatewayManagementApiException making request PostToConnectionRequest to https://8vnardz1z8.execute-api.us-east-2.amazonaws.com/$default. Attempt 1., Amazon.ApiGatewayManagementApi.AmazonApiGatewayManagementApiException: The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.

The Canonical String for this request should have been 'POST /%24default/%2540connections/QNJwacQniYcCGXQ%253D

content-length:0 content-type:application/x-www-form-urlencoded; charset=utf-8 host:8vnardz1z8.execute-api.us-east-2.amazonaws.com user-agent:aws-sdk-dotnet-coreclr/3.7.300.21 ua/2.0 os/windows#10.0.19045.0 md/ARCH#X64 lang/.NET_Core#6.0.25 md/aws-sdk-dotnet-core#3.7.300.21 api/ApiGatewayManagementApi#3.7.300.21 cfg/retry-mode#legacy md/ClientAsync x-amz-api-version:2018-11-29 x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 x-amz-date:20231219T195004Z x-amz-security-token:<>

content-length;content-type;host;user-agent;x-amz-api-version;x-amz-content-sha256;x-amz-date;x-amz-security-token e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'

The String-to-Sign should have been 'AWS4-HMAC-SHA256 20231219T195004Z 20231219/us-east-2/execute-api/aws4_request 37208530f452c3e9254cbd755f807cb6b88f3d9b24fc803e64c9373deeb8227c'

---> Amazon.Runtime.Internal.HttpErrorResponseException: Exception of type 'Amazon.Runtime.Internal.HttpErrorResponseException' was thrown. at Amazon.Runtime.HttpWebRequestMessage.ProcessHttpResponseMessage(HttpResponseMessage responseMessage) at Amazon.Runtime.HttpWebRequestMessage.GetResponseAsync(CancellationToken cancellationToken) at Amazon.Runtime.Internal.HttpHandler1.InvokeAsync[T](IExecutionContext executionContext) at Amazon.Runtime.Internal.Unmarshaller.InvokeAsync[T](IExecutionContext executionContext) at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext) --- End of inner exception stack trace --- at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleExceptionStream(IRequestContext requestContext, IWebResponseData httpErrorResponse, HttpErrorResponseException exception, Stream responseStream) at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleExceptionAsync(IExecutionContext executionContext, HttpErrorResponseException exception) at Amazon.Runtime.Internal.ExceptionHandler1.HandleAsync(IExecutionContext executionContext, Exception exception) at Amazon.Runtime.Internal.ErrorHandler.ProcessExceptionAsync(IExecutionContext executionContext, Exception exception) at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext) at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext) at Amazon.Runtime.Internal.Signer.InvokeAsync[T](IExecutionContext executionContext) at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext) at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext) at Amazon.Runtime.Internal.CredentialsRetriever.InvokeAsync[T](IExecutionContext executionContext) at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)

**IMPORTANT NOTE:** **However, notice that `@connections` is double encoded with `%2540connections/` in both working and not working scenarios.**

**NOTE:** Route is selected based on `Route selection expression` and is usually sent in message JSON, refer [Use wscat to connect to a WebSocket API and send messages to it](https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-how-to-call-websocket-api-wscat.html).

@usix79 Please advise:
- Are you using stage name as `$default`?
- How you are able to use `$default` stage name successfully?
- Could you help us with the end-to-end scenario for testing purposes?

Thanks,
Ashish

[usix79]

@usix79 Good afternoon. I'm unsure if you need to specify $default route as part of ServiceURL. Per Working with stages for HTTP APIs, $default route is served from the base of your API's URL - for example, https://{api_id}.execute-api.{region}.amazonaws.com/. Use @connections commands in your backend service also doesn't demonstrate use of $default for a route. Kindly note that $default is a route, not a stage. The examples I have seen so far construct endpoint based on stage (perhaps using the $default route).

Could you please share the following:

• Result when you invoke PostToConnectionAsync() with endpoint not explicitly specifying $default for a route?
• How is your WebSocket API setup? As per my understanding, one has to define a stage to deploy the API. Have you defined only a single stage? I'm unsure if one could omit a stage in case only one of them is defined.

Thanks, Ashish

Good morning, Ashish I have investigated the issue little bit more and found the following:

1. It appears there was a misconception regarding the $default in AWS API Gateway. $default is the name of a stage as well as a route in my configuration.
2. Altering the stage name by removing the '$' symbol led to a resolution of the issue.
3. To reproduce the bug, one would need to name a stage starting with the '$' symbol.

UPD:

Two-way communication with '$default' route is used In my setup

• WebSocket URL is used on Client: wss://EDITED.execute-api.eu-central-1.amazonaws.com/$default

• @connections URL is used on Backend: https://EDITED.execute-api.eu-central-1.amazonaws.com/$default/@connections

Regarding to usage secenario:

1. Client sends a message to Backend
2. Backend responces immidiately to the request
3. Backend sends an additional message to Client using connectionId from the original request

[ashishdhingra]

To reproduce the bug, one would need to name a stage starting with the '$' symbol

@usix79 Thanks for your inputs. As mentioned in my previous comment, AWS Console allows only $default (with $ character in stage name), otherwise it throws error Stage name only allows a-zA-Z0-9_. In other words, one cannot create a stage with name $test as an example. This aligns with https://docs.aws.amazon.com/apigatewayv2/latest/api-reference/apis-apiid-stages-stagename.html, where it mentions that Stage names can contain only alphanumeric characters, hyphens, and underscores, or be $default.

We are checking with service team if $default should be used when defining stage names (or if it is reserved for some special use case).

Also notice one interesting finding from logs, notice that @connections is double encoded with %2540conne_ctions/ in both working and not working scenarios.. Whereas, it complains when $default is double encoded.

[usix79]

@ashishdhingra, note that double encoding of the '@connections' is considered canonical. We can verify this both by the successful execution of requests and by the error message that was received in case with the '$' in the stage name: The Canonical String for this request should have been 'POST /%24default/%2540connections/QNJwacQniYcCGXQ%253D

[dscpinheiro]

@usix79 Would it be possible to share how your API is set up (for example, with a CloudFormation template)?

We've tried to reproduce the error, but using the mock examples from the API Gateway console (https://github.com/aws/aws-sdk-net/issues/3132#issuecomment-1863382376) has not been enough.

[peterrsongg]

@usix79 I am actively working on this. I was able to reproduce the exact issue so a CF template won't be necessary. The error message you're getting suggests that $default should be single encoded while @connections should be double encoded, which goes against the SigV4 spec here (look for canonicalURI). According to the SIGV4 Spec the canonical URI includes $default which means we should double encode this. Previously we weren't encoding special characters in the URI and we fixed that recently which is leading to this issue.

This service may be one of the only services with a special character in the configured endpoint, so we are trying to determine what is going on on the service side. Thank you for your patience.

[usix79]

@dscpinheiro , We use terraform "aws_apigatewayv2_stage" resourse for setting up the environment (main2.tf.txt). Since we figured out that renaming the stage name solved the issue, we are no longer blocked. Sorry for long delay.

[dscpinheiro]

No worries, thanks for the follow-up and glad you're no longer blocked.

Either way, @peterrsongg has a PR (#3139) queued that should prevent this from happening to other customers (it'll be included in the next SDK release).

Please feel free to re-open this issue if you run into it again.

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.

[peterrsongg]

@usix79 The fix for this issue has been released in Core version 3.7.301.3. You will be able to use $default again in your setup