search

pmem / CacheLib

Pluggable in-process caching engine to build and scale high performance services
https://www.cachelib.org
Apache License 2.0
5 stars 13 forks source link

Background evictor thread #52

Closed byrnedj closed 2 years ago

byrnedj commented 2 years ago

Purpose – To increase insertion path performance, we would like to pre-emptively evict items from the current tier so that allocations may be done without needing to call an eviction.

Method – Implement background thread performing eviction for a given tier (right now this is tier 0 by default but can be extended in the future) – for each memory pool we check the classes to see if the class exceeds a preconfigured free space threshold – if the class does exceed the threshold, we calculate the number of items needed to evict in order to meet that threshold. This way the evictions occur in batch for each class – avoiding contending for the LRU lock for individual evictions.

Results

== Test Results Without Background Evictor Thread == == Allocator Stats == Items in RAM : 115,951 Items in NVM : 0 Alloc Attempts: 4,311,097 Success: 84.89% RAM Evictions : 3,518,361 Background Tier 0 Evictions : 0 Background Tier 0 Eviction Runs : 0 Cache Gets : 4,000,000 Hit Ratio : 44.66% NVM Gets : 0, Coalesced : 100.00% NVM Puts : 0, Success : 0.00%, Clean : 100.00%, AbortsFromDel : 0, AbortsFromGet : 0 NVM Evicts : 0, Clean : 100.00%, Unclean : 0, Double : 0 NVM Deletes : 0 Skipped Deletes: 100.00%

== Throughput for == Total Ops : 4.00 million Total sets: 2,213,563 get : 169,688/s, success : 44.66% set : 93,903/s, success : 100.00% del : 0/s, found : 0.00%

== Test Results With Background Eviction Running every 100ms - 100 allocations per class free == == Allocator Stats == Items in RAM : 28,634 Items in NVM : 0 Alloc Attempts: 4,422,553 Success: 63.16% RAM Evictions : 2,657,768 Background Tier 0 Evictions : 87,098 Background Tier 0 Eviction Runs : 148 Cache Gets : 4,000,000 Hit Ratio : 44.36% NVM Gets : 0, Coalesced : 100.00% NVM Puts : 0, Success : 0.00%, Clean : 100.00%, AbortsFromDel : 0, AbortsFromGet : 0 NVM Evicts : 0, Clean : 100.00%, Unclean : 0, Double : 0 NVM Deletes : 0 Skipped Deletes: 100.00%

== Throughput for == Total Ops : 4.00 million Total sets: 2,225,607 get : 224,013/s, success : 44.36% set : 124,641/s, success : 100.00% del : 0/s, found : 0.00%

Improvement in GET/SET throughput is about 30% for this small test (200MB cache, 4M requests, 400K objects, normal distribution, 8 threads) - see attached configs. config-4GB-DRAM-4GB-DRAM-short.txt config-4GB-DRAM-4GB-DRAM-short-bg-evict.txt

This change is Reviewable

vinser52 commented 2 years ago

Could you please also run your tests with our normal 4GB-DRAM-4GB-PMEM config?

byrnedj commented 2 years ago

Updated results with the normal config after fix with item's not being properly released. About 30% improvement in throughput. bg_10ms_0.001_post.txt no_bg_post.txt

igchor commented 2 years ago

Now, part of: https://github.com/pmem/CacheLib/pull/90