When a new policy is sent to the shipper, it is possible that the new policy will remove configuration that would apply to events that are in the queue (especially disk queue).
Design
When a shipper starts it should save a list of destination data streams it has configuration for. Outputs should check events against the list and log (DEBUG level) and drop any events that it doesn't have configuration for.
When a new policy is received by the shipper, the following should happen:
stop receiving new events from inputs
outputs should stop reading from the queue
wait with deadline for responses from in-flight writes to output to
give them time to complete.
send any ACKs to inputs
reload with new configuration
as part of reload update the list of destination data streams
as part of reload start sending via outputs (which will check against updated list)
as part of reload start reading new events via inputs
Problem Statement
When a new policy is sent to the shipper, it is possible that the new policy will remove configuration that would apply to events that are in the queue (especially disk queue).
Design
When a shipper starts it should save a list of destination data streams it has configuration for. Outputs should check events against the list and log (DEBUG level) and drop any events that it doesn't have configuration for.
When a new policy is received by the shipper, the following should happen:
Other Docs
49