@SqsListener not working after upgrading to Spring Boot 3.1.3

[julpalma]

Type: Bug

Component:

SQS **Describe the bug** Please provide details of the problem, including the version of Spring Cloud that you are using. After upgrading to Spring Boot 3.1.3, @SqsListener is not working anymore. My service is able to send a message to SQS queue but is not receiving the message. I have been trying to fix for more than 3 weeks now, but cant seem to figure this out. Can someone please help me figure out what is missing?? Thank you!!!! Note that the code below is for local testing. With Spring Boot 2.x it works fine. This is my Configuration code, in the Configuration class: ``` @Bean public SqsAsyncClient sqsAsyncClient(){ return SqsAsyncClient.builder().endpointOverride(URI.create("http://localhost:9324")) .region(Region.US_EAST_1) .credentialsProvider(StaticCredentialsProvider.create(new AwsCredentials() { @Override public String accessKeyId() { return "Local"; } @Override public String secretAccessKey() { return "local"; } })) .build(); } @Bean public SqsTemplate sqsTemplateManualContainerInstantiation(SqsAsyncClient sqsAsyncClient) { return SqsTemplate.builder() .sqsAsyncClient(sqsAsyncClient) .build(); } @Bean SqsMessageListenerContainerFactory defaultSqsListenerContainerFactory(SqsAsyncClient sqsAsyncClient) { return SqsMessageListenerContainerFactory .builder() .configure(options -> options .pollTimeout(Duration.ofSeconds(10))) .sqsAsyncClient(sqsAsyncClient) .messageListener((message) -> log.info("Received SQS message {}", message)) .build(); } ``` This is the sendMessage code in the Service class. ``` public void sendMessage(Request request, User user) { try { MyServiceRequest payload = new MyServiceRequest(user, request.getEvent(), TrackingUtil.getTrackingId(), Instant.now().getEpochSecond()); sqsTemplate.send("queueUrl", buildMessageRequest(payload)); } catch (Exception e) { log.error("Failed to send message to SQS for user {}, cause by {}", user, e.getMessage()); } } private SendMessageRequest buildMessageRequest(MyServiceRequest payload) { MessageAttributeValue contentTypeJson = new MessageAttributeValue() .withDataType("String") .withStringValue("application/json"); return new SendMessageRequest("queueUrl", toSqsFormat(payload)) .addMessageAttributesEntry("contentType", contentTypeJson); } private String toSqsFormat(MyServiceRequest payload) { try { return objectMapper.writeValueAsString(payload); } catch (IOException e) { throw new IllegalArgumentException("Failed to convert payload " + message + " to JSON", e); } } ``` This is the listener code: ``` @SqsListener(value = "queueUrl", deletionPolicy = SqsMessageDeletionPolicy.ON_SUCCESS) public void onMessageReceived(MyServiceRequest message) { log.info("Received message from SQS"); } ``` These are the AWS dependencies in the pom file: io.awspring.cloud spring-cloud-aws-messaging 2.4.4 io.awspring.cloud spring-cloud-aws-starter-sqs 3.0.2 When i run the application with debug mode, i see this: SqsAutoConfiguration matched: - @ConditionalOnClass found required classes 'software.amazon.awssdk.services.sqs.SqsAsyncClient', 'io.awspring.cloud.sqs.config.SqsBootstrapConfiguration' (OnClassCondition) - @ConditionalOnProperty (spring.cloud.aws.sqs.enabled=true) matched (OnPropertyCondition) SqsAutoConfiguration#defaultSqsListenerContainerFactory: Did not match: - @ConditionalOnMissingBean (types: io.awspring.cloud.sqs.config.SqsMessageListenerContainerFactory; SearchStrategy: all) found beans of type 'io.awspring.cloud.sqs.config.SqsMessageListenerContainerFactory' defaultSqsListenerContainerFactory (OnBeanCondition) SqsAutoConfiguration#sqsAsyncClient: Did not match: - @ConditionalOnMissingBean (types: software.amazon.awssdk.services.sqs.SqsAsyncClient; SearchStrategy: all) found beans of type 'software.amazon.awssdk.services.sqs.SqsAsyncClient' sqsAsyncClient (OnBeanCondition) SqsAutoConfiguration#sqsTemplate: Did not match: - @ConditionalOnMissingBean (types: io.awspring.cloud.sqs.operations.SqsTemplate; SearchStrategy: all) found beans of type 'io.awspring.cloud.sqs.operations.SqsTemplate' sqsTemplateManualContainerInstantiation (OnBeanCondition) In addition to that, when i check the QueueUrl, i see this:  **Sample** If possible, please provide a test case or sample application that reproduces the problem. This makes it much easier for us to diagnose the problem and to verify that we have fixed it.

[maciejwalkowiak]

You're mixing Spring Cloud AWS 2.x with 3.x. If you're using Spring Boot 3, you must use Spring Cloud AWS 3.x. I am using SQS integration with Spring Boot 3.1.4 so definitely unless there is some unusual config, it does work as expected.

Please try adjusting dependencies and let us know if it helped.

[julpalma]

Thank you so much for the reply @maciejwalkowiak. Understood. I have updated my dependencies to:

io.awspring.cloud spring-cloud-aws-starter-sqs 3.0.2

    <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>spring-messaging</artifactId>
    </dependency>
    <dependency>
        <groupId>software.amazon.awssdk</groupId>
        <artifactId>sqs</artifactId>
        <version>2.20.63</version>
    </dependency>
    <dependency>
        <groupId>com.amazonaws</groupId>
        <artifactId>aws-java-sdk-sqs</artifactId>
    </dependency>
     <dependency>
        <groupId>io.awspring.cloud</groupId>
        <artifactId>spring-cloud-aws-core</artifactId>
        <version>3.0.2</version>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>

I have updated my code:

Configuration:

@Bean
    public SqsTemplate sqsTemplateManualContainerInstantiation() {
        return SqsTemplate.builder()
                .sqsAsyncClient(sqsAsyncClient())
                .build();
    }

    @Bean
    public SqsMessageListenerContainerFactory<Object> defaultSqsListenerContainerFactory() {
        return SqsMessageListenerContainerFactory
                .builder()
                .sqsAsyncClient(sqsAsyncClient())
                .build();
    }

@Bean
    public SqsAsyncClient sqsAsyncClient() {
        return SqsAsyncClient.builder().endpointOverride(URI.create("http://localhost:9324/000000000000/standard"))
                .region(Region.US_EAST_1)
                .credentialsProvider(StaticCredentialsProvider.create(new AwsCredentials() {
                    @Override
                    public String accessKeyId() {
                        return "LOCAL";
                    }

                    @Override
                    public String secretAccessKey() {
                        return "LOCAL";
                    }
                }))
                .build();
    }

Code to sendMessage:

public void sendMessage(Request request, User user) {
try {
MyServiceRequest payload = new MyServiceRequest(user, request.getEvent(), TrackingUtil.getTrackingId(), Instant.now().getEpochSecond());
sqsTemplate.send("queueUrl", toSqsFormat(payload));
} catch (Exception e) {
log.error("Failed to send message to SQS for user {}, cause by {}", user, e.getMessage());
}

private String toSqsFormat(MyServiceRequest message) {
        try {
            return objectMapper.writeValueAsString(message);
        } catch (IOException e) {
            throw new IllegalArgumentException("Failed to convert MyServiceRequest " + message + " to JSON", e);
        }
    }

Code to receive message:

@SqsListener(value = "queueUrl")
public void onMessageReceived(MyServiceRequest message) {
log.info("Received message from SQS");
}

I have imported this SqsBootstrapConfiguration.class as well in the Configuration class.

I am able to send the message to the queue but i have this error:

org.springframework.messaging.converter.MessageConversionException: Could not read JSON: Cannot construct instance of com.amazonaws.event.ProgressListener (no Creators, like default constructor, exist): abstract types either need to be mapped to concrete types, have custom deserializer, or contain additional type information at [Source: (String)"{"requestClientOptions":{"readLimit":131073,"skipAppendUriPath":false},"requestMetricCollector":null,"customRequestHeaders":null,"customQueryParameters":null,"cloneSource":null,"sdkRequestTimeout":null,"sdkClientExecutionTimeout":null,"queueUrl":"http://localhost:9324/000000000000/standard","messageBody":"{\"userId\":\"00000000-0000-0000-0000-000000000000\",\"eventType\":\"call\",\"trackingId\":\"NA_66359720-b056-42ea-b2bd-fc33f9416339\",\"sentAt\":1697828192}","delaySeconds":null,"messageAttr"[truncated 367 chars]; line: 1, column: 810] (through reference chain: com.amazonaws.services.sqs.model.SendMessageRequest["generalProgressListener"]) at org.springframework.messaging.converter.MappingJackson2MessageConverter.convertFromInternal(MappingJackson2MessageConverter.java:235) at org.springframework.messaging.converter.AbstractMessageConverter.fromMessage(AbstractMessageConverter.java:185) at org.springframework.messaging.converter.AbstractMessageConverter.fromMessage(AbstractMessageConverter.java:176) at org.springframework.messaging.converter.CompositeMessageConverter.fromMessage(CompositeMessageConverter.java:57) at io.awspring.cloud.sqs.support.converter.AbstractMessagingMessageConverter.convertPayload(AbstractMessagingMessageConverter.java:172) at io.awspring.cloud.sqs.support.converter.AbstractMessagingMessageConverter.toMessagingMessage(AbstractMessagingMessageConverter.java:153)

Anything else is incorrect or anything else i am missing??

Thank you so much!!!

Juliana

[maciejwalkowiak]

I find it difficult to go through your code samples, formatting could be one of the reasons. Also, com.amazonaws.event.ProgressListener comes from AWS SDK v1.

Are you able to publish a complete project that reproduces your issue? Something I can checkout and run myself?

[julpalma]

Thank you for the reply @maciejwalkowiak. Just updating here with my latest status.

I had this dependency on my pom file:

com.amazonaws

            <artifactId>aws-java-sdk-bom</artifactId>
            <version>1.12.395</version>
           <type>pom</type>
            <scope>import</scope>

I removed and added:

software.amazon.awssdk

            <artifactId>bom</artifactId>
            <version>2.20.63</version>
            <type>pom</type>
            <scope>import</scope>

I see that the Listener container has started successfully now: I see this in the logs: "Container io.awsspring.cloud.sqs.sqsListenerEndpointContainer#0 started"

I dont see that exception anymore.

However, i have this one exception for the token:

11:13:52.722 ERROR --- [nc-response-0-0] i.a.c.s.o.AbstractMessagingTemplate : Error sending message 90593638-c193-7a72-230c-85614f6c7d6a to endpoint http://localhost:9324/000000000000/standard

io.awspring.cloud.sqs.operations.MessagingOperationFailedException: Message send operation failed for message 90593638-c193-7a72-230c-85614f6c7d6a to endpoint http://localhost:9324/000000000000/standard at io.awspring.cloud.sqs.operations.AbstractMessagingTemplate.lambda$sendAsync$11(AbstractMessagingTemplate.java:285) at java.base/java.util.concurrent.CompletableFuture$UniComposeExceptionally.tryFire(CompletableFuture.java:1040) at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2162)

.....

Caused by: software.amazon.awssdk.services.sqs.model.SqsException: The security token included in the request is invalid. (Service: Sqs, Status Code: 403, Request ID: 4c1d683e-87b8-5c5e-817e-358db548edfe) at software.amazon.awssdk.services.sqs.model.SqsException$BuilderImpl.build(SqsException.java:104)

This is my sqsClient:

public SqsAsyncClient sqsAsyncClient() {
        return SqsAsyncClient.builder()
                .region(Region.US_EAST_1)
                .credentialsProvider(StaticCredentialsProvider.create(new AwsCredentials() {
                    @Override
                    public String accessKeyId() {
                        return LOCAL;
                    }

                    @Override
                    public String secretAccessKey() {
                        return LOCAL;
                    }
                }))
                .build();
    }

[julpalma]

And i see this error:

The Canonical String for this request should have been 'POST /

amz-sdk-invocation-id:5c407020-847c-c2a7-c4bc-86917bfee56e amz-sdk-request:attempt=1; max=4 content-length:194 content-type:application/x-www-form-urlencoded; charset=utf-8 host:sqs.us-east-1.amazonaws.com x-amz-date:20231025T181352Z

amz-sdk-invocation-id;amz-sdk-request;content-length;content-type;host;x-amz-date 522127270de60df776f808abc2797e1c1edb0b79baff2350e4b6f62651c3e3d8'

The String-to-Sign should have been 'AWS4-HMAC-SHA256 20231025T181352Z 20231025/us-east-1/sqs/aws4_request c867198790efaa4356d16af8d2bae63bf886f61f84c726d5f94ab42c4ff80b82' (Service: Sqs, Status Code: 403, Request ID: c25fc9d1-af12-55e0-95db-9462f232a84e) at software.amazon.awssdk.services.sqs.model.SqsException$BuilderImpl.build(SqsException.java:104)

[julpalma]

@maciejwalkowiak we found the issue.

The host was set to the real server, when we overwrite, it works fine. Thank you for your help. You can close this issue.

[maciejwalkowiak]

Uff super happy you found it! I am closing it then!

[tmdgusya]

@maciejwalkowiak Thank you for kindness answering for us. Do you happen to have any documents for migrating 2.X to 3.X version related to aws-messaging? because I want to know what classes that I should use when migrate current class to higher version (migrated) classes.