search

poundifdef / smoothmq

An improved drop-in replacement for SQS
https://www.smoothmq.com
GNU Affero General Public License v3.0
2.07k stars 35 forks source link

Message Attributes do not calculate the MD5 hash #35

Open sundbry opened 2 months ago

sundbry commented 2 months ago

I sent a message with a string attribute "content-encoding": "lz4" and the Java SDK threw the invalid MD5 error below:

Caused by: com.amazonaws.AmazonClientException: MD5 returned by SQS does not match the calculation on the original request. (MD5 calculated by the message attributes: "d5b1c8868f6406a7a6a8f4c
ebc4fcc94", MD5 checksum returned: "")
        at com.amazonaws.services.sqs.MessageMD5ChecksumHandler.sendMessageOperationMd5Check(MessageMD5ChecksumHandler.java:120)
        at com.amazonaws.services.sqs.MessageMD5ChecksumHandler.afterResponse(MessageMD5ChecksumHandler.java:80)
        at com.amazonaws.handlers.RequestHandler2Adaptor.afterResponse(RequestHandler2Adaptor.java:49)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.afterResponse(AmazonHttpClient.java:1070)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:818)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697)
        at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561)
        at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:541)
        at com.amazonaws.services.sqs.AmazonSQSClient.doInvoke(AmazonSQSClient.java:3320)
        at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:3286)
        at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:3275)
        at com.amazonaws.services.sqs.AmazonSQSClient.executeSendMessage(AmazonSQSClient.java:2607)

It looks like the SQS server is expected to compute a checksum of the message attributes data and return that to the client for data integrity: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-message-metadata.html#sqs-attributes-md5-message-digest-calculation