noxdafox
commented
1 year ago Hello,
queue-level deduplication does not support persistency of the deduplication headers across restart.
Reason for this is because it's hard to guarantee the queue and deduplication cache to be in sync across reboot. In the past, I was trying to re-load the headers once the node was restarted but the queue implementation does not allow to easily access to the messages once stored on disk.
Therefore, right now, the expected behaviour is that if the node is rebooted message duplicates might occur. This said, the plugin should not crash. Do you have a traceback of the error? How can it be reproduced?
Hi,
first of all thanks for the hard work on the plugin. It is is helping me tremendously:)
I have discovered a situation which may allow duplicate messages to happen in a queue.
I have a queue in which the messages have the right header and are deduplicated. This queue is configured to be persistent. So, when the rabbitmq restarts will reload the messages which were not acknowledged.
The problem I see is that if I try to publish the same messages they will be added in addition to the ones which are already in the queue. If I publish one more time. The second time the messages are not added anymore to the queue. So, what I think is happening is that when the rabbitmq starts and re-add the messages to the queue, they are not added also to the deduplication plugin cash.
would this be hard to fix?