Closed ShooterIT closed 3 years ago
It's really a good question, we expected users to use another disk to store the backup, but seems most users won't do that. So it would be fine only to support the checkpoint for kvrocks backup IMO.
we expected users to use another disk to store the backup
Yes, i think users don't buy this idea, it is not easy to deploy and maintain. For remote backup, that is a good idea because our server originally is a persistent key value server and the backup should not be local. but actually HDFS is not common because some companies don't use it, they may just copy backup to other servers or remote storage.
Maybe it is good idea to support them together, although it is some bit trouble. Firstly, we can only support to use rocksdb checkpoint as kvrocks backup.
yeah, it would be better to support both if it didn't import too much complexity.
For incremental backup, maybe https://github.com/bitleak/kvrocks/wiki/How-to-backup can give you a idea :)
Currently, we use checkpoint of rocksdb to implement kvrocks full synchronization, but kvrocks backup still uses backup of rocksdb. As we know, rocksdb backup costs much bandwidth and space of disk, but rocksdb backup can implement incremental backup and support to store into HDFS, these are good features. So i think we should make a decision to use backup or checkpoint of rocksdb for kvrocks backup. WDYT? @git-hulk @karelrooted @Alfejik