Persist the cluster-enabled status in RocksDB

[git-hulk]

Search before asking

• [X] I had searched in the issues and found no similar issues.

Motivation

Currently, we encode the slot id as the prefix of the key if the cluster mode was enabled according to Kvrocks's configuration. But it would go wrong if we start the Kvrocks with cluster-enabled yes, but change it to cluster-enabled no after restarting.

For the key encoding can see: https://kvrocks.apache.org/community/data-structure-on-rocksdb

Solution

Persist the cluster-enabled configuration in the RocksDB(can use propagate column family) when starting the server, then check if the Kvrocks's cluster-enabled configuration matched the database status.

Are you willing to submit a PR?

• [ ] I'm willing to submit a PR!

[infdahai]

Let's try this.

[git-hulk]

@infdahai Assigned, let's go ahead.

[infdahai]

@git-hulk I find many go tests use the config map including "cluster-enabled": "yes" and the StartServerWithCLIOptions function writes the option to the kvrocks.conf file.

func TestClusterDumpAndLoadClusterNodesInfo(t *testing.T) {
    srv1 := util.StartServer(t, map[string]string{
        "bind":            "0.0.0.0",
        "cluster-enabled": "yes",
    })

in StartServerWithCLIOptions func(), https://github.com/apache/incubator-kvrocks/blob/30e9fd788a4acb585566a008f4d5fff65ee06ce3/tests/gocase/util/server.go#L197-L207

I haven't seen a commad that changes cluster mode, so it means we should define the option in the initial config file if we want to use cluster mode. Restart() will read the kvrocks.conf and set the origin value. https://github.com/apache/incubator-kvrocks/blob/30e9fd788a4acb585566a008f4d5fff65ee06ce3/tests/gocase/util/server.go#L116-L128

change it to cluster-enabled no` after restarting.

so I don't know how this happens, could you explain it?

[git-hulk]

Hi @infdahai

Sorry for missing this comment.

I haven't seen a commad that changes cluster mode, so it means we should define the option in the initial config file if we want to use cluster mode. Restart() will read the kvrocks.conf and set the origin value.

Yeah, that's right that we have no command to set the cluster mode, it must be specified in the configuration file. But the current restart function cannot change the kvrocks.conf, so you need to support overriding the configuration in the restart function first if wanna test this scenario.

[git-hulk]

@inf Are you still working on this?

[infdahai]

@git-hulk I think I know what to do. A new pull request is on the way.

[git-hulk]

Cool, thanks. No hurry, just for asking if you're working on this.

[chrisxu333]

@infdahai are you still working on this? If not I'd like to take over. @git-hulk

[git-hulk]

@chrisxu333 Done, thank you!