Queue Prefix setting is not being considered in the AwsLambdaSQSEndpoint

[vgmello]

Note: This issue description has been updated by Particular Software

Symptoms

When configuring a queue name prefix via the SQS transport configuration option, e.g.:

var endpointConfiguration = new AwsLambdaSQSEndpointConfiguration("ServerlessEndpoint");
endpointConfiguration.Transport.QueueNamePrefix = "someprefix-";

The AWS Lambda endpoint fails to start due to a missing queue. The endpoint tries to access a queue without the configured prefix and will fail if that queue doesn't exist.

Amazon.SQS.Model.QueueDoesNotExistException: The specified queue does not exist for this wsdl version.
 ---> Amazon.Runtime.Internal.HttpErrorResponseException: Exception of type 'Amazon.Runtime.Internal.HttpErrorResponseException' was thrown.
   at Amazon.Runtime.HttpWebRequestMessage.GetResponseAsync(CancellationToken cancellationToken)
   at Amazon.Runtime.Internal.HttpHandler`1.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.Unmarshaller.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.SQS.Internal.ValidationResponseHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
   --- End of inner exception stack trace ---
   at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleExceptionStream(IRequestContext requestContext, IWebResponseData httpErrorResponse, HttpErrorResponseException exception, Stream responseStream)
   at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleExceptionAsync(IExecutionContext executionContext, HttpErrorResponseException exception)
   at Amazon.Runtime.Internal.ExceptionHandler`1.HandleAsync(IExecutionContext executionContext, Exception exception)
   at Amazon.Runtime.Internal.ErrorHandler.ProcessExceptionAsync(IExecutionContext executionContext, Exception exception)
   at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.Signer.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CredentialsRetriever.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.ErrorCallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.MetricsHandler.InvokeAsync[T](IExecutionContext executionContext)
   at NServiceBus.AwsLambdaSQSEndpoint.GetQueueUrl(String queueName) in /_/src/NServiceBus.AwsLambda.SQS/AwsLambdaEndpoint.cs:line 232
   at NServiceBus.AwsLambdaSQSEndpoint.Initialize(AwsLambdaSQSEndpointConfiguration configuration) in /_/src/NServiceBus.AwsLambda.SQS/AwsLambdaEndpoint.cs:line 213
   at NServiceBus.AwsLambdaSQSEndpoint.InitializeEndpointIfNecessary(ILambdaContext executionContext, CancellationToken token) in /_/src/NServiceBus.AwsLambda.SQS/AwsLambdaEndpoint.cs:line 74
   at NServiceBus.AwsLambdaSQSEndpoint.Process(SQSEvent event, ILambdaContext lambdaContext, CancellationToken cancellationToken) in /_/src/NServiceBus.AwsLambda.SQS/AwsLambdaEndpoint.cs:line 41

If the queue exists, this will instead cause errors when trying to acknowledge the message after processing it.

Who's affected

Users of the NServiceBus.AwsLambda.Sqs package that configure a queue name prefix using the QueueNamePrefix setting.

Root cause

This issue is caused by the AwsLambdaSQSEndpointConfiguration not taking into account the specific SQS transport setting when trying to determine the input queue.

Backported to

This bug has been backported to:

• version 1.0.1 via https://github.com/Particular/NServiceBus.AwsLambda.Sqs/pull/313

Original issue content

Hi,

I noticed that the QueueNamePrefix setting is not being used to initialize the AwsLambdaSQSEndpoint. Is this a known issue? Are there any plans to start considering the queue name prefix and potentially the topic name prefix (not tested) for lambda endpoint?

Tks Vitor M

[SzymonPobiega]

Hi

Can you describe your scenario a bit more? With Lambda the settings related to receiving messages (like the queue prefix) are ignored because the input queue is governed by the Lambda settings.

[danielmarbach]

@SzymonPobiega The QueueNamePrefix and the TopicNamePrefix are relevant for outgoing messages too. On top of that since the only way to group things in SQS within the same account is to use prefixes having the ability to specify a prefix would also influence for example these operations

https://github.com/Particular/NServiceBus.AwsLambda.Sqs/blob/master/src/NServiceBus.AwsLambda.SQS/AwsLambdaEndpoint.cs#L179-L180

[evilpilaf]

I'm running into this same issue with version 1.0.0 of the NServiceBus.AwsLambda.SQS package. prefixes aren't being respected which crashes lambda startup.

[mikeminutillo]

@evilpilaf do you get an error message of some kind or does the endpoint just fail to start?

[evilpilaf]

@mikeminutillo I get an error message that says queue does not exist and the name of the queue/endpoint

[SzymonPobiega]

FYI we are starting working on a bugfix for this issue.

[timbussmann]

This bug is fixed as part of the 1.1.1 and 1.0.1 patch releases which should be available soon. Thanks for reporting the issue!

[SzymonPobiega]

The fix has been deployed in 1.1.1 and 1.0.1.