search

vernemq / docker-vernemq

VerneMQ Docker image - Starts the VerneMQ MQTT broker and listens on 1883 and 8080 (for websockets).
https://vernemq.com
Apache License 2.0
177 stars 230 forks source link

fix: add detached script to join cluster #363

Closed mgagliardo91 closed 1 year ago

mgagliardo91 commented 1 year ago

Resolves an issue I was running into related to https://github.com/vernemq/docker-vernemq/issues/304 in which the joining of clusters does not resolve in Kubernetes. After some discovery, it appears that misfire of joining the cluster is likely related to a race condition of the other replicas not being ready to accept the join request. Given that the join request only occurs once, if it misses, it ends up creating a standalone cluster. With a replica set of 3, I was seeing all three sit in their own cluster.

To resolve this, I threw together a script that attempts to join the cluster on an interval after the pod starts up, using the Kube API to recognize that their is other pods to cluster with. The script will attempt to join and check to see if it has successfully joined via the vmq-admin CLI. Once properly joined, it will exit properly. The log of attempts is logged to the /var/log/vernemq/log directory.

ioolkos commented 1 year ago

@mgagliardo91 looks great, thanks! one thing we need to ensure is that this does not affect non-Kubernetes deployments (Swarm, for instance). will review.