Closed vikkio88 closed 5 years ago
Hi, have you figured this out yet?
I haven't worked with this for a while but, when I was actively developing with it, I could push and pop messages without problems.
That said, I'm happy to try to help you debug this. Could you provide me with the configuration you are using to set this up?
hi, I didnt use it anymore, I switched to goaws, I was using the image with the default config, so pushing and reading from the default queue which is already there, but for some reason the receiveMessage was always returning empty.
thanks a lot guys
@roribio I have the same issues. My Example (with NodeJS)
docker-compose.yml
version: '3'
services:
sqs:
ports:
- "9324:9324"
- "9325:9325"
image: roribio16/alpine-sqs:latest
volumes:
- "$PWD/config:/opt/custom"
config/sqs-insight.conf
{
"port": 9325,
"rememberMessages": 100,
"endpoints": [
{
"key": "fake-key",
"secretKey": "fake-secret",
"region": "us-east-1",
"url": "http://localhost:9324/queue/default"
}
],
"dynamicEndpoints": [
{
"key": "fake-key",
"secretKey": "fake-secret",
"region": "us-east-1",
"url": "http://localhost:9324",
"visibility": 1
}
]
}
config/sqs-init.sh
#!/bin/sh
mkdir -p /opt/config
# First, copy default configs:
cp /opt/*.conf /opt/config/
# Secondly, copy custom configs:
cp /opt/custom/*.conf /opt/config/
# Now copy sqs-insight config to correct location:
cp /opt/config/sqs-insight.conf /opt/sqs-insight/config/config_local.json
sleep 1
exit 0
config/elasticmq.conf
include classpath("application.conf")
node-address {
protocol = http
host = "*"
port = 9324
context-path = ""
}
rest-sqs {
enabled = true
bind-port = 9324
bind-hostname = "0.0.0.0"
// Possible values: relaxed, strict
sqs-limits = strict
}
queues {
default {
defaultVisibilityTimeout = 10 seconds
delay = 0 seconds
receiveMessageWait = 0 seconds
},
dev-action {
defaultVisibilityTimeout = 10 seconds
delay = 0 seconds
receiveMessageWait = 0 seconds
}
}
Examples based on the docs. send.js
var queueURL = "http://localhost:9324/queue/dev-action";
var AWS = require('aws-sdk');
AWS.config.update({ region: 'us-east-1' });
var sqs = new AWS.SQS({ apiVersion: '2012-11-05' });
var params = {
DelaySeconds: 10,
MessageAttributes: {
"TitleTitle": {
DataType: "String",
StringValue: "The Whistler"
},
"Author": {
DataType: "String",
StringValue: "John Grisham"
},
"WeeksOn": {
DataType: "Number",
StringValue: "6"
}
},
MessageBody: "Information about current NY Times fiction bestseller for week of 12/11/2016.",
QueueUrl: queueURL
};
sqs.sendMessage(params, function (err, data) {
if (err) {
console.log("Error", err);
} else {
console.log("Success", data.MessageId);
}
});
When I run this script, I get a message in my dev-action
queue:
So that is good !
But now I want to consume it.... receive.js
var queueURL = "http://localhost:9324/queue/dev-action";
// Load the AWS SDK for Node.js
var AWS = require('aws-sdk');
// Set the region
AWS.config.update({region: 'REGION'});
// Create an SQS service object
var sqs = new AWS.SQS({apiVersion: '2012-11-05'});
var params = {
AttributeNames: [
"SentTimestamp"
],
MaxNumberOfMessages: 1,
MessageAttributeNames: [
"Title"
],
QueueUrl: queueURL,
VisibilityTimeout: 20,
WaitTimeSeconds: 0
};
sqs.receiveMessage(params, function(err, data) {
if (err) {
console.log("Receive Error", err);
} else if (data.Messages) {
var deleteParams = {
QueueUrl: queueURL,
ReceiptHandle: data.Messages[0].ReceiptHandle
};
sqs.deleteMessage(deleteParams, function(err, data) {
if (err) {
console.log("Delete Error", err);
} else {
console.log("Message Deleted", data);
}
});
}
});
What I get as (raw) response is (Above script doesn't print anything)
<ReceiveMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/">
<ReceiveMessageResult>
</ReceiveMessageResult>
<ResponseMetadata>
<RequestId>00000000-0000-0000-0000-000000000000</RequestId>
</ResponseMetadata>
</ReceiveMessageResponse>
Any ideas/suggestions ?
Hi,
Thank you for your detailed report 💯
I'm re-opening the issue and I'll investigate further.
Thanks!
I'm testing with these two commands and I can't reproduce your error:
$ aws --endpoint-url http://localhost:9324 sqs send-message --queue-url http://localhost:9324/queue/default --message-body "Hello, World."
$ aws --endpoint-url http://localhost:9324 sqs receive-message --queue-url http://localhost:9324/queue/default
I have two ideas:
defaultVisibilityTimeout
of 10 seconds, which is supposed to prevent two clients accessing the same message concurrently. To test if that's the case, I would exec
into the container and stop sqs-insight
:
$ supervisorctl stop insight
However, I could not reproduce that scenario with the commands I tested with.
defaultVisibilityTimeout = 10 seconds
and your client's VisibilityTimeout: 20
. That could be the source of your grief.Let me know what you find.
Cheers!
P.S.: I accidentally closed the issue 🤷♂️
Closing since there's been no activity in two weeks.
I have two ideas:
Since sqs-insight is retrieving messages continuously, your Javascript client is coming up against the defaultVisibilityTimeout of 10 seconds, which is supposed to prevent two clients accessing the same message concurrently. To test if that's the case, I would exec into the container and stop sqs-insight: $ supervisorctl stop insight
FWIW: I had the exact same issue and this is what was causing the problem
I am having the same issue, and stopping insight
seems to fix it. But I waisted hours before I found this thread.
So wouldn't changing the default value for defaultVisibilityTimeout
to 0 seconds
in opt/elasticmq.conf
fix this issue entirely. And then if anyone wants they can set the timeout in their requests or override it in a mounted volume. It seems strange that the default behavior for the service would be "buggy" in this way.
insight was also the issue for me, i wen around supervisord and used this image in docker-compose with the following parameter
entrypoint: sh -c "/opt/sqs-init.sh && /opt/jdk/bin/java -Dconfig.file=/opt/config/elasticmq.conf -jar /opt/elasticmq-server.jar"
The issue was resolved when the dlq was removed from elasticmq.conf. I created a new file like this https://github.com/roribio/alpine-sqs/blob/master/opt/elasticmq.conf, deleted the dlq queues, and mounted it to /opt/custom. Solved issue where no message was coming back.
@roribio
Stopping the insights fixed the issue for me too. But the sole purpose of the local stack for me is to have queue insights.
Is there a work-around for this to have both insights and have the both aws cli
and aws sdk
retrieving the messages.
Just as a note, I'm using this project to debug Laravel's SQS Queues locally.
Hi, I have been using both this project and the docker-sqs-local, and in both cases I manage to push messages on the queue, using both the php sdk or the awscli, but everytime I try to fetch the messages it comes back empty. Do you know if is something related to this Docker image or is the something that has to do with the latest elasticmq that this project pulls?
Is there a way I can find out?