search

awspring / spring-cloud-aws

The New Home for Spring Cloud AWS
http://awspring.io
Apache License 2.0
859 stars 293 forks source link

Parameter ReceiveRequestAttemptId is invalid for some queue types #579

Closed dk98126 closed 1 year ago

dk98126 commented 1 year ago

Type: Bug

Component: SQS

Describe the bug I wanted to try out the new spring cloud sqs integration using io.awspring.cloud:spring-cloud-aws-starter-sqs:3.0.0-M3. I started docker container softwaremill/elasticmq-native as a local sqs instance for testing purposes. Then I configured simple @SqsListener (the code is provided below). I believe that spring automatically created the specified queue name in that listener annotation, as i started to see it in elasticmq ui.

But then these exceptions started to appear in logs:

2022-12-15T00:28:01.853+03:00 ERROR 46884 --- [nc-response-0-7] i.a.c.s.l.s.AbstractPollingMessageSource : Error polling for messages in queue myQueue

java.util.concurrent.CompletionException: software.amazon.awssdk.services.sqs.model.SqsException: Value e90ad7eb-b4dd-4086-8618-46bc91a030f3 for parameter ReceiveRequestAttemptId is invalid. The request include parameter that is not valid for this queue type (Service: Sqs, Status Code: 400, Request ID: 00000000-0000-0000-0000-000000000000)
    at software.amazon.awssdk.utils.CompletableFutureUtils.errorAsCompletionException(CompletableFutureUtils.java:62) ~[utils-2.18.0.jar:na]
    at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncExecutionFailureExceptionReportingStage.lambda$execute$0(AsyncExecutionFailureExceptionReportingStage.java:51) ~[sdk-core-2.18.0.jar:na]
    at java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:934) ~[na:na]
    at java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:911) ~[na:na]
    at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[na:na]
    at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2162) ~[na:na]
    at software.amazon.awssdk.utils.CompletableFutureUtils.lambda$forwardExceptionTo$0(CompletableFutureUtils.java:76) ~[utils-2.18.0.jar:na]
    at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) ~[na:na]
    at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) ~[na:na]
    at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[na:na]
    at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2162) ~[na:na]
    at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeAttemptExecute(AsyncRetryableStage.java:103) ~[sdk-core-2.18.0.jar:na]
    at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeRetryExecute(AsyncRetryableStage.java:185) ~[sdk-core-2.18.0.jar:na]
    at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.lambda$attemptExecute$1(AsyncRetryableStage.java:171) ~[sdk-core-2.18.0.jar:na]
    at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) ~[na:na]
    at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) ~[na:na]
    at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[na:na]
    at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147) ~[na:na]
    at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.lambda$null$0(MakeAsyncHttpRequestStage.java:105) ~[sdk-core-2.18.0.jar:na]
    at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) ~[na:na]
    at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) ~[na:na]
    at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[na:na]
    at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147) ~[na:na]
    at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.lambda$executeHttpRequest$3(MakeAsyncHttpRequestStage.java:163) ~[sdk-core-2.18.0.jar:na]
    at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) ~[na:na]
    at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) ~[na:na]
    at java.base/java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:482) ~[na:na]
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[na:na]
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[na:na]
    at java.base/java.lang.Thread.run(Thread.java:833) ~[na:na]
Caused by: software.amazon.awssdk.services.sqs.model.SqsException: Value e90ad7eb-b4dd-4086-8618-46bc91a030f3 for parameter ReceiveRequestAttemptId is invalid. The request include parameter that is not valid for this queue type (Service: Sqs, Status Code: 400, Request ID: 00000000-0000-0000-0000-000000000000)
    at software.amazon.awssdk.services.sqs.model.SqsException$BuilderImpl.build(SqsException.java:95) ~[sqs-2.18.0.jar:na]
    at software.amazon.awssdk.services.sqs.model.SqsException$BuilderImpl.build(SqsException.java:55) ~[sqs-2.18.0.jar:na]
    at software.amazon.awssdk.protocols.query.internal.unmarshall.AwsXmlErrorUnmarshaller.unmarshall(AwsXmlErrorUnmarshaller.java:99) ~[aws-query-protocol-2.18.0.jar:na]
    at software.amazon.awssdk.protocols.query.unmarshall.AwsXmlErrorProtocolUnmarshaller.handle(AwsXmlErrorProtocolUnmarshaller.java:102) ~[aws-query-protocol-2.18.0.jar:na]
    at software.amazon.awssdk.protocols.query.unmarshall.AwsXmlErrorProtocolUnmarshaller.handle(AwsXmlErrorProtocolUnmarshaller.java:82) ~[aws-query-protocol-2.18.0.jar:na]
    at software.amazon.awssdk.core.http.MetricCollectingHttpResponseHandler.lambda$handle$0(MetricCollectingHttpResponseHandler.java:52) ~[sdk-core-2.18.0.jar:na]
    at software.amazon.awssdk.core.internal.util.MetricUtils.measureDurationUnsafe(MetricUtils.java:63) ~[sdk-core-2.18.0.jar:na]
    at software.amazon.awssdk.core.http.MetricCollectingHttpResponseHandler.handle(MetricCollectingHttpResponseHandler.java:52) ~[sdk-core-2.18.0.jar:na]
    at software.amazon.awssdk.core.internal.http.async.AsyncResponseHandler.lambda$prepare$0(AsyncResponseHandler.java:89) ~[sdk-core-2.18.0.jar:na]
    at java.base/java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1150) ~[na:na]
    at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[na:na]
    at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147) ~[na:na]
    at software.amazon.awssdk.core.internal.http.async.AsyncResponseHandler$BaosSubscriber.onComplete(AsyncResponseHandler.java:132) ~[sdk-core-2.18.0.jar:na]
    at software.amazon.awssdk.http.nio.netty.internal.ResponseHandler$DataCountingPublisher$1.onComplete(ResponseHandler.java:513) ~[netty-nio-client-2.18.0.jar:na]
    at software.amazon.awssdk.http.nio.netty.internal.ResponseHandler.runAndLogError(ResponseHandler.java:250) ~[netty-nio-client-2.18.0.jar:na]
    at software.amazon.awssdk.http.nio.netty.internal.ResponseHandler.access$600(ResponseHandler.java:75) ~[netty-nio-client-2.18.0.jar:na]
    at software.amazon.awssdk.http.nio.netty.internal.ResponseHandler$PublisherAdapter$1.onComplete(ResponseHandler.java:371) ~[netty-nio-client-2.18.0.jar:na]
    at software.amazon.awssdk.http.nio.netty.internal.nrs.HandlerPublisher.publishMessage(HandlerPublisher.java:402) ~[netty-nio-client-2.18.0.jar:na]
    at software.amazon.awssdk.http.nio.netty.internal.nrs.HandlerPublisher.flushBuffer(HandlerPublisher.java:338) ~[netty-nio-client-2.18.0.jar:na]
    at software.amazon.awssdk.http.nio.netty.internal.nrs.HandlerPublisher.receivedDemand(HandlerPublisher.java:291) ~[netty-nio-client-2.18.0.jar:na]
    at software.amazon.awssdk.http.nio.netty.internal.nrs.HandlerPublisher.access$200(HandlerPublisher.java:61) ~[netty-nio-client-2.18.0.jar:na]
    at software.amazon.awssdk.http.nio.netty.internal.nrs.HandlerPublisher$ChannelSubscription$1.run(HandlerPublisher.java:495) ~[netty-nio-client-2.18.0.jar:na]
    at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174) ~[netty-common-4.1.85.Final.jar:4.1.85.Final]
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167) ~[netty-common-4.1.85.Final.jar:4.1.85.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470) ~[netty-common-4.1.85.Final.jar:4.1.85.Final]
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:566) ~[netty-transport-4.1.85.Final.jar:4.1.85.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[netty-common-4.1.85.Final.jar:4.1.85.Final]
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.85.Final.jar:4.1.85.Final]
    ... 1 common frames omitted

This specific part of this exception is the most interesting: "Value e90ad7eb-b4dd-4086-8618-46bc91a030f3 for parameter ReceiveRequestAttemptId is invalid. The request include parameter that is not valid for this queue type"

I think the cause of this problem is that ReceiveRequestAttemptId is only applicable for FIFO queue. But thats just a guess based on documentation in software.amazon.awssdk.services.sqs.model.ReceiveMessageRequest.Builder#receiveRequestAttemptId(String) which says "This parameter applies only to FIFO (first-in-first-out) queues.".

Sample

  1. That is build.gradle.kts file.
import org.jetbrains.kotlin.gradle.tasks.KotlinCompile

plugins {
    id("org.springframework.boot") version "3.0.0"
    id("io.spring.dependency-management") version "1.1.0"
    kotlin("jvm") version "1.7.21"
    kotlin("plugin.spring") version "1.7.21"
}

group = "com.github.dk98126"
version = "0.0.1-SNAPSHOT"
java.sourceCompatibility = JavaVersion.VERSION_17

repositories {
    mavenCentral()
    maven {
        url = uri("https://repo.spring.io/milestone")
    }
}

dependencies {
    implementation("io.awspring.cloud:spring-cloud-aws-starter-sqs:3.0.0-M3")
    implementation("org.jetbrains.kotlin:kotlin-reflect")
    implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8")
    testImplementation("org.springframework.boot:spring-boot-starter-test")
}

tasks.withType<KotlinCompile> {
    kotlinOptions {
        freeCompilerArgs = listOf("-Xjsr305=strict")
        jvmTarget = "17"
    }
}

tasks.withType<Test> {
    useJUnitPlatform()
}

  1. This is application.properties file. These are default properties for elastiqmq docker container.

    spring.cloud.aws.sqs.endpoint=http://localhost:9324
spring.cloud.aws.sqs.region=elasticmq
spring.cloud.aws.credentials.access-key=x
spring.cloud.aws.credentials.secret-key=x

  2. And main class file. I have no additional configuration files.

    
package com.github.dk98126.springcloudaws3tryout

import io.awspring.cloud.sqs.annotation.SqsListener import org.springframework.boot.autoconfigure.SpringBootApplication import org.springframework.boot.runApplication

@SpringBootApplication class SpringCloudAws3TryoutApplication {

@SqsListener("myQueue")
fun listen(message: String) {
    System.out.println(message);
}

}

fun main(args: Array) { runApplication(*args) }

tomazfernandes commented 1 year ago

Hi @dk98126, that's an known issue with elasticmq: https://github.com/awspring/spring-cloud-aws/issues/529

We should fix this, but in the meantime you can target localstack or AWS itself and it'll work as expected.

Thanks.

dk98126 commented 1 year ago

@tomazfernandes Thank you!