search

apache / kvrocks-controller

Apache Kvrocks Controller is a cluster management tool for Apache Kvrocks.
https://kvrocks.apache.org/
Apache License 2.0
78 stars 42 forks source link

network with 100% packet loss and process hang scenarios cannot switch quickly #105

Closed bbqccx closed 10 months ago

bbqccx commented 11 months ago

Based on our test results, in scenarios where the master node experiences 100% packet loss and process hang, it cannot switch quickly. Each probe takes about 1 minute to time out, which causes a 15 * 3 second switch to take 15 minutes to occur。 suggest setting the timeout time for each probe to 2 to 3 seconds, so that even in extreme scenarios, problems can be detected as early as possible and control can initiate the switch proactively。

git-hulk commented 11 months ago

@bbqccx Thanks for your feedback, will fix it soon.

bbqccx commented 10 months ago

readTimeout = 3 time.Second writeTimeout = 3 time.Second

test ok

git-hulk commented 10 months ago

Thank you!