Can't receive messages

[vikkio88]

Hi, I have been using both this project and the docker-sqs-local, and in both cases I manage to push messages on the queue, using both the php sdk or the awscli, but everytime I try to fetch the messages it comes back empty. Do you know if is something related to this Docker image or is the something that has to do with the latest elasticmq that this project pulls?

Is there a way I can find out?

[roribio]

Hi, have you figured this out yet?

I haven't worked with this for a while but, when I was actively developing with it, I could push and pop messages without problems.

That said, I'm happy to try to help you debug this. Could you provide me with the configuration you are using to set this up?

[vikkio88]

hi, I didnt use it anymore, I switched to goaws, I was using the image with the default config, so pushing and reading from the default queue which is already there, but for some reason the receiveMessage was always returning empty.

[vikkio88]

thanks a lot guys

[SegersIan]

@roribio I have the same issues. My Example (with NodeJS)

Configuration

docker-compose.yml

version: '3'
services:
  sqs:
    ports:
    - "9324:9324"
    - "9325:9325"
    image: roribio16/alpine-sqs:latest
    volumes:
    - "$PWD/config:/opt/custom"

config/sqs-insight.conf

{
    "port": 9325,
    "rememberMessages": 100,

    "endpoints": [
        {
            "key": "fake-key",
            "secretKey": "fake-secret",
            "region": "us-east-1",
                "url": "http://localhost:9324/queue/default"
        }
    ],

    "dynamicEndpoints": [
        {
           "key": "fake-key",
           "secretKey": "fake-secret",
           "region": "us-east-1",
           "url": "http://localhost:9324",
           "visibility": 1
        }
    ]
}

config/sqs-init.sh

#!/bin/sh

mkdir -p /opt/config

# First, copy default configs:
cp /opt/*.conf /opt/config/

# Secondly, copy custom configs:
cp /opt/custom/*.conf /opt/config/

# Now copy sqs-insight config to correct location:
cp /opt/config/sqs-insight.conf /opt/sqs-insight/config/config_local.json

sleep 1
exit 0

config/elasticmq.conf

include classpath("application.conf")

node-address {
    protocol = http
    host = "*"
    port = 9324
    context-path = ""
}

rest-sqs {
    enabled = true
    bind-port = 9324
    bind-hostname = "0.0.0.0"
    // Possible values: relaxed, strict
    sqs-limits = strict
}

queues {
    default {
        defaultVisibilityTimeout = 10 seconds
        delay = 0 seconds
        receiveMessageWait = 0 seconds
    },
    dev-action {
        defaultVisibilityTimeout = 10 seconds
        delay = 0 seconds
        receiveMessageWait = 0 seconds
    }
}

Code

Examples based on the docs. send.js

var queueURL = "http://localhost:9324/queue/dev-action";
var AWS = require('aws-sdk');
AWS.config.update({ region: 'us-east-1' });
var sqs = new AWS.SQS({ apiVersion: '2012-11-05' });

var params = {
    DelaySeconds: 10,
    MessageAttributes: {
        "TitleTitle": {
            DataType: "String",
            StringValue: "The Whistler"
        },
        "Author": {
            DataType: "String",
            StringValue: "John Grisham"
        },
        "WeeksOn": {
            DataType: "Number",
            StringValue: "6"
        }
    },
    MessageBody: "Information about current NY Times fiction bestseller for week of 12/11/2016.",
    QueueUrl: queueURL
};

sqs.sendMessage(params, function (err, data) {
    if (err) {
        console.log("Error", err);
    } else {
        console.log("Success", data.MessageId);
    }
});

When I run this script, I get a message in my dev-action queue:

So that is good !

But now I want to consume it.... receive.js

var queueURL = "http://localhost:9324/queue/dev-action";

// Load the AWS SDK for Node.js
var AWS = require('aws-sdk');
// Set the region
AWS.config.update({region: 'REGION'});

// Create an SQS service object
var sqs = new AWS.SQS({apiVersion: '2012-11-05'});

var params = {
    AttributeNames: [
        "SentTimestamp"
    ],
    MaxNumberOfMessages: 1,
    MessageAttributeNames: [
        "Title"
    ],
    QueueUrl: queueURL,
    VisibilityTimeout: 20,
    WaitTimeSeconds: 0
};

sqs.receiveMessage(params, function(err, data) {
    if (err) {
        console.log("Receive Error", err);
    } else if (data.Messages) {
        var deleteParams = {
            QueueUrl: queueURL,
            ReceiptHandle: data.Messages[0].ReceiptHandle
        };
        sqs.deleteMessage(deleteParams, function(err, data) {
            if (err) {
                console.log("Delete Error", err);
            } else {
                console.log("Message Deleted", data);
            }
        });
    }
});

What I get as (raw) response is (Above script doesn't print anything)

<ReceiveMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/">
              <ReceiveMessageResult>

              </ReceiveMessageResult>
              <ResponseMetadata>
                <RequestId>00000000-0000-0000-0000-000000000000</RequestId>
              </ResponseMetadata>
            </ReceiveMessageResponse>

Any ideas/suggestions ?

[roribio]

Hi,

Thank you for your detailed report 💯

I'm re-opening the issue and I'll investigate further.

Thanks!

[roribio]

I'm testing with these two commands and I can't reproduce your error:

$ aws --endpoint-url http://localhost:9324 sqs send-message --queue-url http://localhost:9324/queue/default --message-body "Hello, World."

$ aws --endpoint-url http://localhost:9324 sqs receive-message --queue-url http://localhost:9324/queue/default

I have two ideas:

1. Since sqs-insight is retrieving messages continuously, your Javascript client is coming up against the defaultVisibilityTimeout of 10 seconds, which is supposed to prevent two clients accessing the same message concurrently. To test if that's the case, I would exec into the container and stop sqs-insight:

$ supervisorctl stop insight

However, I could not reproduce that scenario with the commands I tested with.

2. There's a mismatch between the queue's defaultVisibilityTimeout = 10 seconds and your client's VisibilityTimeout: 20. That could be the source of your grief.

Let me know what you find.

Cheers!

P.S.: I accidentally closed the issue 🤷‍♂️

[roribio]

Closing since there's been no activity in two weeks.

[kwarcu]

I have two ideas:

Since sqs-insight is retrieving messages continuously, your Javascript client is coming up against the defaultVisibilityTimeout of 10 seconds, which is supposed to prevent two clients accessing the same message concurrently. To test if that's the case, I would exec into the container and stop sqs-insight: $ supervisorctl stop insight

FWIW: I had the exact same issue and this is what was causing the problem

[maldimirov]

I am having the same issue, and stopping insight seems to fix it. But I waisted hours before I found this thread.

So wouldn't changing the default value for defaultVisibilityTimeout to 0 seconds in opt/elasticmq.conf fix this issue entirely. And then if anyone wants they can set the timeout in their requests or override it in a mounted volume. It seems strange that the default behavior for the service would be "buggy" in this way.

[ruslantalpa]

insight was also the issue for me, i wen around supervisord and used this image in docker-compose with the following parameter

entrypoint: sh -c "/opt/sqs-init.sh && /opt/jdk/bin/java -Dconfig.file=/opt/config/elasticmq.conf -jar /opt/elasticmq-server.jar"

[sc6]

The issue was resolved when the dlq was removed from elasticmq.conf. I created a new file like this https://github.com/roribio/alpine-sqs/blob/master/opt/elasticmq.conf, deleted the dlq queues, and mounted it to /opt/custom. Solved issue where no message was coming back.

[Olcod]

@roribio

Stopping the insights fixed the issue for me too. But the sole purpose of the local stack for me is to have queue insights.

Is there a work-around for this to have both insights and have the both aws cli and aws sdk retrieving the messages.

Just as a note, I'm using this project to debug Laravel's SQS Queues locally.