apache / incubator-pegasus

Apache Pegasus - A horizontally scalable, strongly consistent and high-performance key-value store
https://pegasus.apache.org/
Apache License 2.0
1.99k stars 312 forks source link

feat(local_partition_split): Add local_partition_split CLI tool #1930

Closed acelyc111 closed 8 months ago

acelyc111 commented 8 months ago

This patch introduces a new CLI tool local_partition_split, which is used to split the local partitions offline.

Similar to the online partition split feature, it's helpful to split the table which has large amount of data but with a few partitions into more partitions to improve the throughput and lower latency.

Besides, this tool is possible to be used on older Pegasus versions, if you want to do that, it's necessary to cherr-pick this patch to the corresponding older branch and rebuild the tool, then this tool will use corresponding version of RocksDB library to generate new replicas.