search

awslabs / amazon-sqs-java-extended-client-lib

An extension to the Amazon SQS client that enables sending and receiving messages up to 2GB via Amazon S3.
Apache License 2.0
211 stars 109 forks source link

Client adds a message attribute name to ReceiveMessageRequest for every call #25

Open dsw2127 opened 6 years ago

dsw2127 commented 6 years ago

AmazonSQSExtendedClient.receiveMessage(ReceiveMessageRequest) always appends SQSExtendedClientConstants.RESERVED_ATTRIBUTE_NAME to the list of message attribute names of the request passed in. Therefore if the request object is re-used for multiple invocations, this request can grow and grow. Eventually, if allowed to grow over time, SQS will deny the request with a 413 Request Entity Too Large. This seems to happen when the header gets to be somewhere around 1MB.

com.amazonaws.services.sqs.model.AmazonSQSException: null (Service: AmazonSQS; Status Code: 413; Error Code: 413 Request Entity Too Large; Request ID: null)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1588)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1258)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1030)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:742)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:716)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
        at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
        at com.amazonaws.services.sqs.AmazonSQSClient.doInvoke(AmazonSQSClient.java:1792)
        at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:1768)
        at com.amazonaws.services.sqs.AmazonSQSClient.executeReceiveMessage(AmazonSQSClient.java:1374)
        at com.amazonaws.services.sqs.AmazonSQSClient.receiveMessage(AmazonSQSClient.java:1350)
        at com.amazon.sqs.javamessaging.AmazonSQSExtendedClientBase.receiveMessage(AmazonSQSExtendedClientBase.java:217)
        at com.amazon.sqs.javamessaging.AmazonSQSExtendedClient.receiveMessage(AmazonSQSExtendedClient.java:349)

Example of message attribute names over four invocations:


MessageAttributeNames: [SQSLargePayloadSize]
MessageAttributeNames: [SQSLargePayloadSize, SQSLargePayloadSize]
MessageAttributeNames: [SQSLargePayloadSize, SQSLargePayloadSize, SQSLargePayloadSize]
MessageAttributeNames: [SQSLargePayloadSize, SQSLargePayloadSize, SQSLargePayloadSize, SQSLargePayloadSize]
dsw2127 commented 6 years ago

Workaround is to not re-use (i.e. create a new) ReceiveMessageRequest object for every invocation.

robin-aws commented 6 years ago

Hi there! Thanks for contributing this fix. We have pulled it into the latest release: https://github.com/awslabs/amazon-sqs-java-extended-client-lib/releases/tag/1.0.2