search

awspring / spring-cloud-aws

The New Home for Spring Cloud AWS
http://awspring.io
Apache License 2.0
882 stars 300 forks source link

Facing issue with POJO Deserialization in SqsListener #820

Closed ankurraiyani closed 1 year ago

ankurraiyani commented 1 year ago

I am using spring-cloud-aws-starter-sqs, version 3.0.1 with spring-boot-starter-parent, version 3.0.6

I have just 2 classes in my sample project. The main application class looks like below

@SpringBootApplication
public class SpringAwsSqsApplication {

    public static void main(String[] args) {
        ApplicationContext cntx = SpringApplication.run(SpringAwsSqsApplication.class, args);
        AwsSqsService sqsSer = cntx.getBean(AwsSqsService.class);
        sqsSer.send();
    }

}

The AWS SQS Service class looks like below

@Service
@Log4j2
public class AwsSqsService {

    @Value("${aws.queue}")
    private String awsQueue;

    @Autowired
    private SqsTemplate sqsTemp;

    public void send() {
        sqsTemp.send(awsQueue, new SampleRecord("Hello"));
    }

    @SqsListener({"${aws.queue}"})
    public void read(SampleRecord message) {
        log.info("Msg Recd => " + message);
    }

}

The application.properties is as below

aws.queue=new-queue
spring.cloud.aws.credentials.access-key=xxxxxxxxx
spring.cloud.aws.credentials.secret-key=xxxxxxxxx
spring.cloud.aws.region.static=us-east-1

The message is being sent successfully but for SqsListner I am getting below error message. I tried difference solutions available from different articles but most of them are either for version 2.x.x or not solving the issue.

Caused by: com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'SampleRecord': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
 at [Source: (String)"SampleRecord(name=Hello)"; line: 1, column: 13]

Full StackTrace is as below

java.util.concurrent.CompletionException: io.awspring.cloud.sqs.listener.AsyncAdapterBlockingExecutionFailedException: Error executing action in BlockingMessageListenerAdapter
        at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:332) ~[na:na]
        at java.base/java.util.concurrent.CompletableFuture.uniApplyNow(CompletableFuture.java:674) ~[na:na]
        at java.base/java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:662) ~[na:na]
        at java.base/java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:2200) ~[na:na]
        at io.awspring.cloud.sqs.listener.pipeline.MessageListenerExecutionStage.process(MessageListenerExecutionStage.java:49) ~[spring-cloud-aws-sqs-3.0.1.jar:3.0.1]
        at io.awspring.cloud.sqs.listener.pipeline.MessageProcessingPipelineBuilder$ComposingMessagePipelineStage.lambda$process$0(MessageProcessingPipelineBuilder.java:80) ~[spring-cloud-aws-sqs-3.0.1.jar:3.0.1]
        at java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1187) ~[na:na]
        at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2341) ~[na:na]
        at io.awspring.cloud.sqs.listener.pipeline.MessageProcessingPipelineBuilder$ComposingMessagePipelineStage.process(MessageProcessingPipelineBuilder.java:80) ~[spring-cloud-aws-sqs-3.0.1.jar:3.0.1]
        at io.awspring.cloud.sqs.listener.pipeline.MessageProcessingPipelineBuilder$FutureComposingMessagePipelineStage.process(MessageProcessingPipelineBuilder.java:104) ~[spring-cloud-aws-sqs-3.0.1.jar:3.0.1]
        at io.awspring.cloud.sqs.listener.pipeline.MessageProcessingPipelineBuilder$FutureComposingMessagePipelineStage.process(MessageProcessingPipelineBuilder.java:104) ~[spring-cloud-aws-sqs-3.0.1.jar:3.0.1]
        at io.awspring.cloud.sqs.listener.pipeline.MessageProcessingPipelineBuilder$FutureComposingMessagePipelineStage.process(MessageProcessingPipelineBuilder.java:104) ~[spring-cloud-aws-sqs-3.0.1.jar:3.0.1]
        at io.awspring.cloud.sqs.listener.pipeline.MessageProcessingPipelineBuilder$FutureComposingMessagePipelineStage.process(MessageProcessingPipelineBuilder.java:104) ~[spring-cloud-aws-sqs-3.0.1.jar:3.0.1]
        at io.awspring.cloud.sqs.listener.sink.AbstractMessageProcessingPipelineSink.lambda$execute$0(AbstractMessageProcessingPipelineSink.java:99) ~[spring-cloud-aws-sqs-3.0.1.jar:3.0.1]
        at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1768) ~[na:na]
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[na:na]
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[na:na]
        at java.base/java.lang.Thread.run(Thread.java:1623) ~[na:na]
Caused by: io.awspring.cloud.sqs.listener.AsyncAdapterBlockingExecutionFailedException: Error executing action in BlockingMessageListenerAdapter
        at io.awspring.cloud.sqs.listener.AsyncComponentAdapters$AbstractThreadingComponentAdapter.wrapWithBlockingException(AsyncComponentAdapters.java:162) ~[spring-cloud-aws-sqs-3.0.1.jar:3.0.1]
        at io.awspring.cloud.sqs.listener.AsyncComponentAdapters$AbstractThreadingComponentAdapter.runInSameThread(AsyncComponentAdapters.java:124) ~[spring-cloud-aws-sqs-3.0.1.jar:3.0.1]
        at io.awspring.cloud.sqs.listener.AsyncComponentAdapters$AbstractThreadingComponentAdapter.execute(AsyncComponentAdapters.java:111) ~[spring-cloud-aws-sqs-3.0.1.jar:3.0.1]
        at io.awspring.cloud.sqs.listener.AsyncComponentAdapters$BlockingMessageListenerAdapter.onMessage(AsyncComponentAdapters.java:208) 
~[spring-cloud-aws-sqs-3.0.1.jar:3.0.1]
        ... 14 common frames omitted
Caused by: io.awspring.cloud.sqs.listener.ListenerExecutionFailedException: Listener failed to process messages 8a6fd73f-9fbe-4d41-8d5e-57dc8ddf7012
        at io.awspring.cloud.sqs.listener.adapter.AbstractMethodInvokingListenerAdapter.createListenerException(AbstractMethodInvokingListenerAdapter.java:79) ~[spring-cloud-aws-sqs-3.0.1.jar:3.0.1]
        at io.awspring.cloud.sqs.listener.adapter.AbstractMethodInvokingListenerAdapter.createListenerException(AbstractMethodInvokingListenerAdapter.java:83) ~[spring-cloud-aws-sqs-3.0.1.jar:3.0.1]
        at io.awspring.cloud.sqs.listener.adapter.AbstractMethodInvokingListenerAdapter.invokeHandler(AbstractMethodInvokingListenerAdapter.java:59) ~[spring-cloud-aws-sqs-3.0.1.jar:3.0.1]
        at io.awspring.cloud.sqs.listener.adapter.MessagingMessageListenerAdapter.onMessage(MessagingMessageListenerAdapter.java:41) ~[spring-cloud-aws-sqs-3.0.1.jar:3.0.1]
        at io.awspring.cloud.sqs.listener.AsyncComponentAdapters$BlockingMessageListenerAdapter.lambda$onMessage$0(AsyncComponentAdapters.java:208) ~[spring-cloud-aws-sqs-3.0.1.jar:3.0.1]
        at io.awspring.cloud.sqs.listener.AsyncComponentAdapters$AbstractThreadingComponentAdapter.runInSameThread(AsyncComponentAdapters.java:120) ~[spring-cloud-aws-sqs-3.0.1.jar:3.0.1]
        ... 16 common frames omitted
Caused by: org.springframework.messaging.converter.MessageConversionException: Could not read JSON: Unrecognized token 'SampleRecord': was 
expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
 at [Source: (String)"SampleRecord(name=Hello)"; line: 1, column: 13]
        at org.springframework.messaging.converter.MappingJackson2MessageConverter.convertFromInternal(MappingJackson2MessageConverter.java:235) ~[spring-messaging-6.0.8.jar:6.0.8]
        at org.springframework.messaging.converter.AbstractMessageConverter.fromMessage(AbstractMessageConverter.java:185) ~[spring-messaging-6.0.8.jar:6.0.8]
        at org.springframework.messaging.converter.CompositeMessageConverter.fromMessage(CompositeMessageConverter.java:70) ~[spring-messaging-6.0.8.jar:6.0.8]
        at org.springframework.messaging.handler.annotation.support.PayloadMethodArgumentResolver.resolveArgument(PayloadMethodArgumentResolver.java:141) ~[spring-messaging-6.0.8.jar:6.0.8]
        at org.springframework.messaging.handler.invocation.HandlerMethodArgumentResolverComposite.resolveArgument(HandlerMethodArgumentResolverComposite.java:118) ~[spring-messaging-6.0.8.jar:6.0.8]
        at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.getMethodArgumentValues(InvocableHandlerMethod.java:147) ~[spring-messaging-6.0.8.jar:6.0.8]
        at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:115) ~[spring-messaging-6.0.8.jar:6.0.8]
        at io.awspring.cloud.sqs.listener.adapter.AbstractMethodInvokingListenerAdapter.invokeHandler(AbstractMethodInvokingListenerAdapter.java:56) ~[spring-cloud-aws-sqs-3.0.1.jar:3.0.1]
        ... 19 common frames omitted
Caused by: com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'SampleRecord': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
 at [Source: (String)"SampleRecord(name=Hello)"; line: 1, column: 13]
        at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:2418) ~[jackson-core-2.14.2.jar:2.14.2]
        at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:759) ~[jackson-core-2.14.2.jar:2.14.2]    
        at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._reportInvalidToken(ReaderBasedJsonParser.java:3038) ~[jackson-core-2.14.2.jar:2.14.2]
        at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._handleOddValue(ReaderBasedJsonParser.java:2079) ~[jackson-core-2.14.2.jar:2.14.2]
        at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:805) ~[jackson-core-2.14.2.jar:2.14.2]
        at com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:4817) ~[jackson-databind-2.14.2.jar:2.14.2]       
        at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4723) ~[jackson-databind-2.14.2.jar:2.14.2]      
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3677) ~[jackson-databind-2.14.2.jar:2.14.2]
        at org.springframework.messaging.converter.MappingJackson2MessageConverter.convertFromInternal(MappingJackson2MessageConverter.java:230) ~[spring-messaging-6.0.8.jar:6.0.8]
        ... 26 common frames omitted

pom.xm file is as below

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>3.0.6</version>
        <relativePath /> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.websopti.aws.sqs</groupId>
    <artifactId>spring-aws-sqs</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>spring-aws-sqs</name>
    <description>Demo project for Spring Boot</description>
    <properties>
        <java.version>17</java.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-mail</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>io.awspring.cloud</groupId>
            <artifactId>spring-cloud-aws-starter-sqs</artifactId>
            <version>3.0.1</version>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <configuration>
                    <excludes>
                        <exclude>
                            <groupId>org.projectlombok</groupId>
                            <artifactId>lombok</artifactId>
                        </exclude>
                    </excludes>
                </configuration>
            </plugin>
        </plugins>
    </build>

</project>

Please guide and let me know if any additional information is required

tomazfernandes commented 1 year ago

Hi @ankurraiyani.

This is odd, we have plenty of tests asserting this behavior works.

Can you share more of your stack trace? It would also help if you could create a minimal sample project that reproduces the behavior you're seeing.

Thanks.

maciejwalkowiak commented 1 year ago

@ankurraiyani do you use DevTools by any chance?

ankurraiyani commented 1 year ago

Yes I do have that dependency. @maciejwalkowiak I've updated the question and added the pom.xml file

ankurraiyani commented 1 year ago

Hi @tomazfernandes

I have updated the stacktrace in question and here I have attached the project as well which is POC. Please let me know if you need any additional details.

Thanks spring-aws-sqs.zip

ankurraiyani commented 1 year ago

@maciejwalkowiak @tomazfernandes Have you get a chance to look into this issue?

maciejwalkowiak commented 1 year ago

I am pretty sure this issue is caused by Devtools. Unfortunately this is not something we figured out how to solve. Can you try running your code without devtools?

ankurraiyani commented 1 year ago

Hey @maciejwalkowiak !

I have commented out the spring-boot-devtools dependency and yes it worked ! 😄

maciejwalkowiak commented 1 year ago

I am really sorry it works that way. I got also burned by this problem. Spring Boot Devtools is sometimes disappointing - it works until it doesn't. We have similar issue with DynamoDB.

cmetzlerhg commented 10 months ago

Are there any updates? I ran into the same issue with dev-tools enabled. Would it work using the SqsTemplate to receive messages instead of the @SqsListener?

maciejwalkowiak commented 10 months ago

No updates. If you figure out a workaround - contributions welcome - either as a code change or guidelines in docs.

tomazfernandes commented 10 months ago

Would it work using the SqsTemplate to receive messages instead of the @SqsListener?

It should work, but I'd definitely advise against changing production code for DevTool's sake.