jodydonetti
commented
1 day ago Hi @jarodriguez-itsoft ,
I'm doing some work on this in the new v2.
The idea is not to require a manual setting, but to make it automatic based on concrete usage: I'm already doing this for memory entries (eg: cache entries in L1, the memory level), whereby I only populate the metadata if it's needed, based on the features you are using like eager refresh & co. This means that if you are not using any feature that requires metadata, metadata will not be there.
I've never done this for the L2 (distributed level) for... reasons 🤔 ... but I'm getting back to this again.
The short version is that in v2, metadata will be there only if needed.
Will update on this.
Problem
I’m using this amazing library to manage both combined (in-memory and distributed) caches and distributed-only caches.
Some of these caches hold millions of entries, accessed concurrently by dozens of app nodes (following a producer/consumer pattern). Thanks to a new feature introduced in recent versions of the library, we can now use distributed-only caches, which has spared us from having nodes consume massive amounts of memory unnecessarily.
So far, so good, but KeyDB is also consuming a lot of memory, and we aim to reduce its usage as well.
The issue is that FusionCache stores a lot of metadata in Redis, which is essentially useless since Redis already handles expiration, and the in-memory part of the cache is disabled.
For example, to simply store the string value "var", we get something like this:
127.0.0.1:6379> HGETALL "v0:foo" 1) "absexp" 2) "638710876575684414" 3) "data" 4) "{\"$id\":\"1\",\"v\":\"bar\",\"m\":{\"$id\":\"2\",\"e\":\"2024-12-29T16:47:37.5660416+00:00\"},\"t\":638684956575645632}" 5) "sldexp" 6) "-1"
That's quite too much memory overhead when the caches have millions of entries.
Solution
Add a cache-level (or entry-level option) to control, at the distributed level, if the value should be serialized/deserialized as usual or simply stored/retrieved in its raw form.
127.0.0.1:6379> GET "foo" "bar"