search

opensearch-project / k-NN

🆕 Find the k-nearest neighbors (k-NN) for your vector data
https://opensearch.org/docs/latest/search-plugins/knn/index/
Apache License 2.0
156 stars 123 forks source link

Perform cache maintenance in a separate thread #2239

Open jmazanec15 opened 3 weeks ago

jmazanec15 commented 3 weeks ago

Description

For the NativeMemoryCacheManager, we allow users to enable an "expirary" time setting, so that cache entries that arent touched for some duration, get marked as expired and are then evicted.

However, the problem is that the guava cache does not maintain a separate thread that can perform this maintenance. Instead it will typically perform this maintenance on writes and sometimes on reads. See https://github.com/google/guava/wiki/CachesExplained#timed-eviction. Thus, if you are not touching the cache for awhile, entries may be expired but not get evicted. Thus, resources are not freed.

The Guava team recommends creating a separate thread that will call cache.cleanUp() to explicitly perform maintenance. This will ensure that the time-based evictions happen as one would expect.

luyuncheng commented 3 weeks ago

LGTM

navneet1v commented 3 weeks ago

@jmazanec15 we should do the cache cleanup for Quantization state cache too.