This extends the behavior of the current vespalib::cache class to support 2-segment caching. Default behavior is non-segmented LRU, maintaining current cache semantics and expected run-time overhead.
The cache can optionally be constructed (or configured) with a secondary cache capacity; this automatically enables SLRU semantics. When SLRU is enabled, cached elements are present in exactly one out of two segments; probationary or protected.
Newly cached entries (from write-throughs or reads) are placed at the head of the probationary segment. Tail entries evicted from this segment are lost to the sands of time. If a probationary element is explicitly read it is moved to the head of the protected segment. This may cause the capacity of the protected segment to be exceeded, causing one or more protected tail entries to be evicted. Entries evicted from the protected segment are reinserted at the head of the probationary segment, giving them a second chance. This reinsertion may in turn evict entries from the probationary segment tail. The circle of life. 🦁
SLRU involves juggling entries between two separate LRUs, which has higher overhead than a simple single LRU. This means that cache performance will likely decrease slightly if the cache is already large enough to fit all data (in which case eviction policies do not matter because they will not be used in practice). The opposite is expected to be the case if the cache is substantially smaller than the size of all cacheable objects.
Note that the regular non-SLRU cache is implemented to reside entirely within the probationary segment.
@havardpe please review 👀
This extends the behavior of the current
vespalib::cacheclass to support 2-segment caching. Default behavior is non-segmented LRU, maintaining current cache semantics and expected run-time overhead.The cache can optionally be constructed (or configured) with a secondary cache capacity; this automatically enables SLRU semantics. When SLRU is enabled, cached elements are present in exactly one out of two segments; probationary or protected.
Newly cached entries (from write-throughs or reads) are placed at the head of the probationary segment. Tail entries evicted from this segment are lost to the sands of time. If a probationary element is explicitly read it is moved to the head of the protected segment. This may cause the capacity of the protected segment to be exceeded, causing one or more protected tail entries to be evicted. Entries evicted from the protected segment are reinserted at the head of the probationary segment, giving them a second chance. This reinsertion may in turn evict entries from the probationary segment tail. The circle of life. 🦁
SLRU involves juggling entries between two separate LRUs, which has higher overhead than a simple single LRU. This means that cache performance will likely decrease slightly if the cache is already large enough to fit all data (in which case eviction policies do not matter because they will not be used in practice). The opposite is expected to be the case if the cache is substantially smaller than the size of all cacheable objects.
Note that the regular non-SLRU cache is implemented to reside entirely within the probationary segment.