search

gchq / stroom

Stroom is a highly scalable data storage, processing and analysis platform.
https://gchq.github.io/stroom-docs/
Apache License 2.0
425 stars 56 forks source link

Stroom Proxy - Ability to drain #4311

Open gcdev373 opened 3 weeks ago

gcdev373 commented 3 weeks ago

When shutting down a Stroom proxy instance, it is often necessary to drain its data to downstream Stroom endpoints.

This is most important when the system is going down. And it is vital before shutting down a Stroom Proxy instance permanently.

Stroom Proxy should have an option that it automatically attempts to drain on shutdown. And ideally be distributed with a separate script that drains proxy and returns a zero exit code only when complete, to assist scripting.

gcdev373 commented 1 week ago

There might be a bit of a hole when a feed status changes to a REJECT or DROP status from RECEIVE. Ideally, any events already received at Proxy could still be forwarded to Stroom. Might be worth considering at same time as this issue.