Tencent / Tendis

Tendis is a high-performance distributed storage system fully compatible with the Redis protocol.
http://tendis.cn
Other
2.87k stars 317 forks source link

Refine TTL support for string type(key-value) to reduce compaction read bandwidth #229

Open leonliao opened 1 year ago

leonliao commented 1 year ago

Description

Disk read IO bandwidth is high even with blob db enabled(KV separation feature for RocksDB).

Expected Behavior

Avoid using value to store TTL, then avoid the high disk read bandwidth.

Current Behavior

For blob enabled RocksDB, current Tendis implementation reads value to check TTL in KVTtlCompactionFilter, which causes very high unnecessary disk read bandwidth. E.g., the size of a value for a key is 30KB, during compaction, Tendis just needs the 8 bytes TTL in value prepended to value.

Possible Solution

Using separate key to store TTL for KV type.

Context

2.5.0-rocksdb-v6.23.3

takenliu commented 5 months ago

可能还是会有一些问题吧。比如,什么时候决定写这个特殊的ttl呢?怎么删除它呢?KVTtlCompactionFilter里面是否要再读取这个ttl?