duplicated emitted messages after a fail recover

ptgoetz / storm-jms

Storm JMS Integration

Apache License 2.0

78 stars 66 forks source link

duplicated emitted messages after a fail recover #16

Open frascuchon opened 10 years ago

frascuchon commented 10 years ago

Hi,

Reading the documentation of the method session.recover(), I think the recovery process after a failure causes messages inside within the topology (after spout), are discarded by jms queue and re-send to spout. This causes duplicate messages to be processed.

Maybe an explicit emit after fail event can solve this problem (with some kind of retry policy)

I open this issue to discuss a possible solution.

Thanks in advance!

albertocsm commented 9 years ago

As I see it, exactly once delivery guarantees are not to be expected. I certainly don't. Usualy Its a lot easier to have a idempotent block on your pipeline (eg. http://camel.apache.org/idempotent-consumer.html)