Closed dfa1 closed 2 months ago
I am seeing the same in our envs since updating to 9.0.0. It does spam quite a lot on environments with thousands of SST files, on each DB open.
It was introduced here https://github.com/facebook/rocksdb/pull/11406 It's not a problem, it's just some backward compatibility stuff:
For backward compatibility, we fall back to TailPrefetchStats and last to simple heuristics that the tail size is a linear portion of the file size - see PR conversation for more.
I was thinking of opening PR to suggest changing log level to INFO
, so I'll do that
or to know what to do, like documentation
You'll want to make sure your files are compacted by a new enough version before we delete the complicated original behavior as a fallback (TailPrefetchStats). Any version that spews the warning is new enough.
@ajkr thanks!
I triggered a manual compaction via Java API:
public void compactRange() throws RocksDBException {
compactRange(null);
}
but I still see a lot logs like that (RocksDB 8.11.4). Any idea why?
Because compactRange()
without any options won't compact every single SST.
I think BottommostLevelCompaction::kForce
option in CompactRangeOptions::bottommost_level_compaction
should help. With this API:
public void compactRange(
/* @Nullable */ final ColumnFamilyHandle columnFamilyHandle,
final byte[] begin, final byte[] end,
final CompactRangeOptions compactRangeOptions) throws RocksDBException {
compactRange(nativeHandle_,
begin, begin == null ? -1 : begin.length,
end, end == null ? -1 : end.length,
compactRangeOptions.nativeHandle_,
columnFamilyHandle == null ? 0 : columnFamilyHandle.nativeHandle_);
}
problem has been fixed by using compactRange()
with bottommost_level_compaction
thanks @ajkr @andlr!
Expected behavior
To not receive these logs (or to know what to do, like documentation).
Actual behavior
Many logs per second![image](https://github.com/facebook/rocksdb/assets/5798698/c46ed71e-59b9-499f-a2d1-4d91a54b4d5f)
Steps to reproduce the behavior
I'm not able to reproduce it locally (windows 11) but this appears often in our Linux servers. It is happening since the upgrade to rocksdbjni 8.x.
This is the OPTIONS file: