Detect broken replica when moving a group of replicas

openark / orchestrator

MySQL replication topology management and HA

Apache License 2.0

5.61k stars 927 forks source link

Open dveeden opened 7 years ago

dveeden commented 7 years ago

Relate to #234

Orchestrator waits indefinitely hen running -c relocate-replicas with one or more broken replicas.

Orchestrator should detect the broken replica via a timeout or based on health (replica was 'black' in the gui) and then either:

not start the operation
complete the operation without the troublesome replica (and show a message about it)
abort the operation and restore replication

shlomi-noach commented 7 years ago

To clarify, this happens because the replica's TCP connection itself was hanging, correct?