OpenAtomFoundation / pikiwidb

a high-performance, large-capacity, multi-tenant, data-persistent, strong data consistency based on raft, Redis-compatible elastic KV data storage system based on RocksDB
BSD 3-Clause "New" or "Revised" License
202 stars 63 forks source link

feat:add event listerner #175

Closed hotjump closed 7 months ago

hotjump commented 8 months ago
  1. 将braft log index 和 rocksdbsequence在内存里关连起来,memtable在flush时调用回调函数将与之相关的log index写到sst文件里的table properties block。
  2. 1中增加步长选项,比如每100个存一对关系,牺牲准确性但是可以大幅减少内存使用。
  3. 重启或者故障恢复时从sst文件中拿到每个CF的最大log index(使用大于1的步长,这个log index可能略小于其对应的真实的log index导致最多多on_apply 100条日志),在on_apply时可以跳过已经apply过的日志,可以显著缩短启动时间。
  4. 增加event listener对维护的list进行purge
AlexStocks commented 8 months ago

等常远 binlog

Issues-translate-bot commented 8 months ago

Bot detected the issue body's language is not English, translate it automatically.


Wait for Changyuan binlog

AlexStocks commented 8 months ago

https://github.com/OpenAtomFoundation/pikiwidb/pull/166 pr 合并