ben-manes
commented
9 years ago The negative GC impact of weak/soft reference application caches is significant to warrant most usages abusive. A more proper example would be to externalize to disk or off-heap. In your example, it would be better to provide additional diagnostics to determine the optimal cache configuration (size, timeouts) and improved eviction policy than to use heap-based layered caches.
The cache follows the rule of not executing foreign code under its own locks, as emphasized in Doug Lea's Concurrent Programming in Java. This and to reduce hold times is the reason that the removal listener is asynchronous. If Guava supported Java 8's compute methods, it is easy to decorate to provide synchronous listeners. For example, see Caffeine's JSR-107 (JCache) adapter where in-order synchronous listeners are required by the spec.
There's a lot more I could go into regarding the complexity and concerns of this enhancement. In short, I think its proper for external contributions to provide these features in addition to Guava, rather than as part of it. If the compute methods were available then these features would not require any Guava changes. It would be smarter to provide the minimal building blocks than the features themselves.
MarionaDSR
tfij
It would be great if guava had a multi-level cache - a cache degrading from hard references -> soft references -> weak references, with each level configurable as a regular cache.
A single-level cache doesn't opportunistically cache more than is specified if there is the memory free to do so - you either specify a fixed number or fixed timeout, then it is removed from the cache. A multi-level cache would keep values evicted from the hard cache in memory using a soft or weak reference so that it is still cached (and available) if there is the free memory, but can be garbage collected if the GC has to.
Similar to specifying an access timeout, values could be 'promoted' back up the cache levels if they are accessed (from weak -> soft -> hard), so there is a degree of effort to make frequently accessed values more likely to be in memory.
The naive implementation of chaining together separate LocalCaches doesn't work because the RemovalListener is invoked lazily - if a value is evicted, it is put onto the removal queue, but it may be some time before the listeners are invoked, during which time the entry is not available in the cache.
Alternatively, a helpful option to add to LocalCache would be to invoke RemovalListener deterministically - so that as soon as a value is evicted, RemovalListener is notified to add it to the next caching level without it being left in limbo in the removal queue.