Initial State:
Control 1 is not running.
Control 2 is running.
Operations:
1.Start Control 1 as a follower.
2.Kill Control 2 after a few hours. It was found that Control 1 did not take over and crashed .
3.Restart Control 1 as the leader and start Control 1 as a follower
4.Retry killing Control 2, and it was found that Control 1 could take over this time
Control1 log:
panic: close of closed channel
goroutine 1 [running]:
github.com/RocksLabs/kvrocks_controller/controller/migrate.(Migrator).Shutdown(0xc000124f00)
/home/k v rock s/controller/kvrocks_controller/controller/migrate/migrator.go:79 +0x25
github.com/RocksLabs/kvrocks_controller/controller.(Controller).unloadModules(0xc000124f50)
/home/kvrocks/controller/kvrocks_controller/controller/controller.go:92 +0x45
github.com/RocksLabs/kvrocks_controller/controller.(Controller).Stop.func1()
/home/kvrocks/controller/kvrocks_controller/controller/controller.go:181 +0xcf
sync.(Once).doSlow(0xc0004a6a20?, 0xc000044238?)
/usr/local/go/src/sync/once.go:74 +0xc2
sync.(Once).Do(...)
/usr/local/go/src/sync/once.go:65
github.com/RocksLabs/kvrocks_controller/controller.(Controller).Stop(0xbf6085?)
/home/kvrocks/controller/kvrocks_controller/controller/controller.go:175 +0x45
github.com/RocksLabs/kvrocks_controller/server.(*Server).Stop(0xc00033e4b0)
/home/kvrocks/controller/kvrocks_controller/server/server.go:115 +0x45
main.main()
/home/kvrocks/controller/kvrocks_controller/cmd/server/main.go:101 +0x6c5
nohup: ignoring input
{"level":"info","ts":1693818282.8894243,"caller":"server/main.go:75","msg":"Kvrocks controller is running with version: unstable"}
{"level":"info","ts":1693818283.0661163,"caller":"controller/controller.go:118","msg":"Lost the leader campaign, will unload modules"}
{"level":"info","ts":1693818421.2138064,"caller":"controller/controller.go:109","msg":"Start as the leader"}
Initial State: Control 1 is not running. Control 2 is running.
Operations: 1.Start Control 1 as a follower. 2.Kill Control 2 after a few hours. It was found that Control 1 did not take over and crashed . 3.Restart Control 1 as the leader and start Control 1 as a follower 4.Retry killing Control 2, and it was found that Control 1 could take over this time
Control1 log:
panic: close of closed channel goroutine 1 [running]: github.com/RocksLabs/kvrocks_controller/controller/migrate.(Migrator).Shutdown(0xc000124f00) /home/k v rock s/controller/kvrocks_controller/controller/migrate/migrator.go:79 +0x25 github.com/RocksLabs/kvrocks_controller/controller.(Controller).unloadModules(0xc000124f50) /home/kvrocks/controller/kvrocks_controller/controller/controller.go:92 +0x45 github.com/RocksLabs/kvrocks_controller/controller.(Controller).Stop.func1() /home/kvrocks/controller/kvrocks_controller/controller/controller.go:181 +0xcf sync.(Once).doSlow(0xc0004a6a20?, 0xc000044238?) /usr/local/go/src/sync/once.go:74 +0xc2 sync.(Once).Do(...) /usr/local/go/src/sync/once.go:65 github.com/RocksLabs/kvrocks_controller/controller.(Controller).Stop(0xbf6085?) /home/kvrocks/controller/kvrocks_controller/controller/controller.go:175 +0x45 github.com/RocksLabs/kvrocks_controller/server.(*Server).Stop(0xc00033e4b0) /home/kvrocks/controller/kvrocks_controller/server/server.go:115 +0x45 main.main() /home/kvrocks/controller/kvrocks_controller/cmd/server/main.go:101 +0x6c5 nohup: ignoring input {"level":"info","ts":1693818282.8894243,"caller":"server/main.go:75","msg":"Kvrocks controller is running with version: unstable"} {"level":"info","ts":1693818283.0661163,"caller":"controller/controller.go:118","msg":"Lost the leader campaign, will unload modules"} {"level":"info","ts":1693818421.2138064,"caller":"controller/controller.go:109","msg":"Start as the leader"}