search

gchq / stroom

Stroom is a highly scalable data storage, processing and analysis platform.
https://gchq.github.io/stroom-docs/
Apache License 2.0
431 stars 55 forks source link

Improved cluster management #2121

Open stroomdev66 opened 3 years ago

stroomdev66 commented 3 years ago

We need it to be easier to add/remove nodes from a cluster.

Possibly use zookeeper/curator for things like leader election, cluster locking, etc. (gh-2070). This may also allow for more dynamic role allocation to nodes, e.g. up to two nodes need role X which is assigned on a first come first served basis.

at055612 commented 2 years ago

Also worth considering Hazlecast for its distributed locks/semaphores/etc. Probably easier to integrate into stroom. Would also give us lots of other useful tools, e.g. distributed state. https://docs.hazelcast.com/imdg/4.0/data-structures/distributed-data-structures.