Redis HA - Githubissues

rogeruiz commented 7 years ago

Taking a further look at this. We're blocked, unless we cherry-pick, on this until #53 is merged in because that PR contains the relevant StatefulSets changes to the broker.

LinuxBozo commented 7 years ago

@rogeruiz rebase on that branch?

rogeruiz commented 7 years ago

@cnelson good catch. yeah I forgot to update it. We aren't even publishing a latest 😁

rogeruiz commented 7 years ago

Also, I think this is ready to be merged in since we're moving on to acceptance tests in the cg-deploy-kubernetes repo now.

I've verified all this manually for now but am hoping to cover this in the acceptance tests.

[x] deployment of broker works
[x] cf create-service & cf delete-service work as expected
[x] Persistent data volume mounted at /data
[x] Custom docker builds for all three images redis-sentinel-proxy, redis-sentinel-init, & redis-server-init.
[x] failover works successfully using Sentinel when servers are removed manually.
[x] failover works successfully using Sentinel during a deployment of k8s.

While writing this, something happened to the k8s cluster on dev during a restart. looks like Sentinels didn't have enough time to elect a new master because both the server-* and two sentinels went away during a deployment and haven't been able to recover properly.

rogeruiz commented 7 years ago

Had a typo under matchLabels which is now fixed. @jmcarp this look good to merge?

rogeruiz commented 7 years ago

cool thanks. pipeline and secrets flown and stored respectively.

cloud-gov / kubernetes-broker

Redis HA #55