The naive mark/sweep behavior of the cache's GC technically allows nested cache usage but it'll drop the intermediate results at the end of the first revision where the topmost cached value is used.
Previously I'd thought about installing a new cache in the local context for the duration of the init/with operations and GC'ing it only when that local cache is accessed. This has very nice "edge oriented" mutations and incremental cost but it makes it very difficult for nested cache usage to share the same backing store. This is more problematic for use cases like the web where we're best off interning lots of strings all over the place. If nested cache values causes those strings to be duplicated in memory, it's quite possible we'd lose the benefit of caching them in the first place.
Now my thinking is leaning more towards traditional GC approaches and only tracking the liveness of "roots" -- i.e. a topmost query that returned in the given revision. Nested cache values need to have that dependency tracked up to a live root or be dropped.
The main drawback I see is that implementing this will add significant complexity and testing obligations. It's possible it could introduce new pathological performance behavior, although I think that generally the ability to cache more aggressively will outweigh that.
anp
commented
4 years ago
The naive mark/sweep behavior of the cache's GC technically allows nested cache usage but it'll drop the intermediate results at the end of the first revision where the topmost cached value is used.
Previously I'd thought about installing a new cache in the local context for the duration of the
init/withoperations and GC'ing it only when that local cache is accessed. This has very nice "edge oriented" mutations and incremental cost but it makes it very difficult for nested cache usage to share the same backing store. This is more problematic for use cases like the web where we're best off interning lots of strings all over the place. If nested cache values causes those strings to be duplicated in memory, it's quite possible we'd lose the benefit of caching them in the first place.Now my thinking is leaning more towards traditional GC approaches and only tracking the liveness of "roots" -- i.e. a topmost query that returned in the given revision. Nested cache values need to have that dependency tracked up to a live root or be dropped.
The main drawback I see is that implementing this will add significant complexity and testing obligations. It's possible it could introduce new pathological performance behavior, although I think that generally the ability to cache more aggressively will outweigh that.