search

apache / kvrocks-controller

Apache Kvrocks Controller is a cluster management tool for Apache Kvrocks.
https://kvrocks.apache.org/
Apache License 2.0
78 stars 42 forks source link

Parallel send the probe request in the same cluster #158

Closed git-hulk closed 2 months ago

git-hulk commented 2 months ago

Before this patch, the controller will sequence send the probe request in the same cluster, and the following nodes will be delayed if some of them are blocking.

I have tested the new codes and it will parallel send the probe request, and the failover also works well:

{"level":"info","timestamp":"2024-04-09T20:59:25.690+0800","caller":"probe/cluster.go:127","msg":"probe the cluster node: 127.0.0.1:6666","id":"tR3AfdspqUHCzCqks4OjHQgJHpErweDqSw5Ryw2Q","role":"master","addr":"127.0.0.1:6666"}
{"level":"info","timestamp":"2024-04-09T20:59:25.700+0800","caller":"probe/cluster.go:127","msg":"probe the cluster node: 127.0.0.1:6667","id":"XocfSzHxyW6hHpeK9GrkCPoxdAeDMSqRLF0FErAZ","role":"slave","addr":"127.0.0.1:6667"}
{"level":"info","timestamp":"2024-04-09T20:59:30.695+0800","caller":"probe/cluster.go:127","msg":"probe the cluster node: 127.0.0.1:6666","id":"tR3AfdspqUHCzCqks4OjHQgJHpErweDqSw5Ryw2Q","role":"master","addr":"127.0.0.1:6666"}
{"level":"info","timestamp":"2024-04-09T20:59:30.695+0800","caller":"probe/cluster.go:127","msg":"probe the cluster node: 127.0.0.1:6667","id":"XocfSzHxyW6hHpeK9GrkCPoxdAeDMSqRLF0FErAZ","role":"slave","addr":"127.0.0.1:6667"}
{"level":"info","timestamp":"2024-04-09T20:59:35.690+0800","caller":"probe/cluster.go:127","msg":"probe the cluster node: 127.0.0.1:6667","id":"XocfSzHxyW6hHpeK9GrkCPoxdAeDMSqRLF0FErAZ","role":"slave","addr":"127.0.0.1:6667"}
{"level":"info","timestamp":"2024-04-09T20:59:35.691+0800","caller":"probe/cluster.go:127","msg":"probe the cluster node: 127.0.0.1:6666","id":"tR3AfdspqUHCzCqks4OjHQgJHpErweDqSw5Ryw2Q","role":"master","addr":"127.0.0.1:6666"}
{"level":"info","timestamp":"2024-04-09T20:59:40.688+0800","caller":"probe/cluster.go:127","msg":"probe the cluster node: 127.0.0.1:6666","id":"tR3AfdspqUHCzCqks4OjHQgJHpErweDqSw5Ryw2Q","role":"master","addr":"127.0.0.1:6666"}
{"level":"info","timestamp":"2024-04-09T20:59:40.689+0800","caller":"probe/cluster.go:127","msg":"probe the cluster node: 127.0.0.1:6667","id":"XocfSzHxyW6hHpeK9GrkCPoxdAeDMSqRLF0FErAZ","role":"slave","addr":"127.0.0.1:6667"}
git-hulk commented 2 months ago

It's error-prone to test manually while changing the failover/migration, I have added an issue to improve this.

bbqccx commented 2 months ago

so strong