facebook / rocksdb

A library that provides an embeddable, persistent key-value store for fast storage.
http://rocksdb.org
GNU General Public License v2.0
28.23k stars 6.26k forks source link

how to compute lifetime of SSTable? #12863

Open zhenyoung opened 1 month ago

zhenyoung commented 1 month ago

In my study(level compcation style), I define the lifetim of SSTable as the time interval between two compactions.

definition

For example, the first compaction output the sst1, and the timestamp of this time is its created_time. When the second compaction is triggered which selects sst1 as one of victim in this compaction, and the timestamp of this time is its death_time. So the result of death_time - created_time is the lifetime of this sst1.

I have added two fields created_time and death_time to the table_properties.h

doubts

To record created_time, I have add the following code in BlockBasedTableBuilder::WritePropertiesBlock() of block_based_table_builder.cc while writing properties block to storage.

int64_t _current_time = 0;
auto status = rep_->ioptions.clock->GetCurrentTime(&_current_time);
const uint64_t current_time = static_cast<uint64_t>(_current_time);
rep_->props.created_time = current_time;
rep_->props.death_time = 0;

image

And I want to record death_time at the end of PickCompaction() in compaction_picker_level.cc.

Compaction* c = GetCompaction();
// added code
...

I want to get information from this c instance and set death_time field.

But I'm not sure whether the properties of SSTable can be modified or not this time. And how to write code to set this death_time property?

ajkr commented 1 month ago

For creation time, we have file_creation_time in the MANIFEST AddFile entry. For deletion time, you could approximate it using the latest file_creation_time seen before the DeleteFile entry. That's what https://github.com/ZonedStorage/rocksdb_trace_translator does.