Closed slobo closed 7 years ago
There are 2 other problems except for concurrent creation of queues:
ConfigObject
doesn't keep entries order - typesafehub/config#365List
used for createQueues
in ElasticMQServerConfig
doesn't keep order of itemsInstead of using ordering, we could just create any queues without deadletter queues and then do the ones with deadletter queues. Do we NEED to do them in order?
Can a dead-letter queue have own dead-letter queue? In that case we would need to calculate a dependancy graph.
In Perl land I've used Graph::topological_sort, I'm sure there is something similar for JVM?
(This of course assumes we don't want to allow any cycles - does that restriction exist in SQS?)
Following @ferrants suggestion, I have commented out exception thrown when dead letter queue doesn't exists. From quick test it looks like everything is working properly.
Changes: https://github.com/adamw/elasticmq/compare/master...mkubenka:fix/102-deadletter-queue-dependency
@slobo i think a dlq can have another dlq, though I don't think it's a good move, architecturally speaking. can't think of any good cases for it. certainly cycles are not standard either.
@mkubenka , i think that not caring if a queue's dlq exists at the time of creation is a reasonable way to go. it'll push the error to when you're using the queues instead of at creation time. this would allow us to not concern ourselves with creation order. 👍
We use dead letter queues with another dead letter queue. Our primary queue polls N times, before dropping to an error dead letter queue where statuses get updated. It then drops to the next dead letter queue.
Yeah, i think there are a number of interesting things to do with chaining DLQs, i.e. you could implement an exponential back-off retry mechanism by stringing together a bunch of queues with different visibility timeouts.
Ideally, config file should be validated for any queue name misspellings, but if possible for ElasticMQ to just delay DLQ connection until the time it actually has to transfer into it, I think it's a good solution for now.
Fixed in 0.13.3
As per #100, queues should not be created concurrently, instead they should either follow config file order, or in order that will satisfy deadletter queue dependencies. @adamw suggest problem is here: https://github.com/adamw/elasticmq/blob/release-0.13.2/server/src/main/scala/org/elasticmq/server/ElasticMQServer.scala#L63
Example failure:
conf:
log: