Flume events reinjection must be done not before a certain configurable time

[frbattid]

Current reinjection mechanism depends on the number of already existent Flume events within the channel:

• If such a number is large, then the next retrie will be done after a long time.
• If such a number id short, then the next retrie will be too soon, exhausting the retries number very quick.

The first behaviour is hard to manage since we should be able to manipulate the order withn the queue. Nevertheless, the second behaviour is very easy to fix if a reinjection timestamp header is added when a Flume event is reinjected and the sinks check if a certain configurable time has passed from that timestamp.

Effort: 1 man day

[frbattid]

An interesting parameter associated to the Flume channels is the keep-alive one. It specifies the number of seconds an event put or take operation is definitely considered as a failed operation. I think this is a parameter added in Flume 1.5.0, thus Cygnus should update to this version if wanted to be used as a complement for our TTL mechanism.

[frbattid]

This issue has no sense. It has no sense to control when the reinjected events are going to be persisted because Cygnus will try to persist many other new events and they will fail as well.

Closing.