Allow to skip storing the value in memory cache

[marafiq]

Is your feature request related to a problem? Please describe. Application A sets a key value but never tries to get it. Instead, application B gets and removes the cache value and also would like to use the memory cache as the key will be accessed multiple times. Since the fusion cache allows the backplane notifications, when the cache value is changed by Application A, then Application B will refresh its value in the memory cache.

In other words, one process sets the value, and another process gets and removes the cache value.

Describe the solution you'd like A flag SkipMemoryCache on entry options like SkipDistributedCache so the memory cache can be skipped while setting it. It will be set to true in another process, thus using the memory cache.

Alternate Setting the duration 0, and distributed duration to the desired time, might work.

Additional Context In some cases, when the data is large in size you might want to skip the memory cache altogether but still be able to take advantage of backplane notifications and other features.

[jodydonetti]

Hi @marafiq and thanks for using FusionCache!

That is an interesting scenario to consider: let me think about it a little bit and I'll come back to you with my thoughts.

[CM2Walki]

I have the same feature request, but a slightly different setup: I'm using an Azure Function and to stick to @marafiq's terminology:

Application A is a TimerTrigger (a cron job) that periodically regenerates a list of large cache values. These are set using IFusionCache.SetAsync.

Application B is a HttpTrigger (a web server route), that serves those cache values using IFusionCache.GetOrSetAsync.

---

The problem

The worker executing the TimerTrigger ends up with a MemoryCache of ~6GB whilst the DistributedCache (Redis) is only at ~3GB, possibly because of compression. This is why I only want to serve these large values directly from Redis, even at the expense of latency.

I've tried setting:

options.Duration = TimeSpan.Zero;
options.IsFailSafeEnabled = false;
options.DistributedCacheDuration = TimeSpan.FromDays(1);

for IFusionCache.SetAsync but the worker executing the TimerTrigger always ends up with the same 6GB of memory usage, whilst the workers only serving directly from the distributed cache are at less than 1GB.

I would also propose a SkipMemoryCache entry flag or to find out the reason why setting an instant MemoryCache expire doesn't free the underlying memory.

---

This is related to #59, as Azure Functions when not self-hosted have a memory limit of ~1.5GB, being able to fine-tune the usages of the MemoryCache is important. You get billed for cpu and memory time.

[celluj34]

I also have a similar request. We run some database calls inside GetOrSetAsync, wrapped in a try/catch block. In case the database call fails and hits our catch block, we set the Duration to TimeSpan.FromTicks(1). However we're seeing some exceptions because FusionCache sees that as in the past by the time the code actually gets to checking that duration.

If we had a way to say "Actually, don't cache this value because it's not good" that would be really helpful for us.

[celluj34]

Now that we have context.Modified and context.NotModified perhaps we could get context.Ignore or context.DoNotCache or something like that?

[jodydonetti]

Hi @celluj34

I also have a similar request. We run some database calls inside GetOrSetAsync, wrapped in a try/catch block. In case the database call fails and hits our catch block, we set the Duration to TimeSpan.FromTicks(1). However we're seeing some exceptions because FusionCache sees that as in the past by the time the code actually gets to checking that duration.

I'll check this out, will let you know.

One question though: is fail-safe enabled in this case? I'm asking because even by handling TimeSpan.Zero natively, I cannot avoid saving it in the cache if fail-safe is enabled, because it would actually mean something like "save it as already expired, but keep it available for future problems as a fallback".

Makes sense?

In case fail-safe is disabled or can be disabled, I can natively handle zero as a special case, like it's already done for infinity.

If we had a way to say "Actually, don't cache this value because it's not good" that would be really helpful for us. I was about to tell you to just not catch the exception, but that may trigger fail-safe (if enabled) which may not be what you want.

The problem I see is that GetOrSet() MUST return a value, so in the case of not wanting to return anything, what should FusionCache return?

In other words: if you want it to throw, you can avoid catching the exception (or re-throw it), and if you want to return some some of default value with a low duration, you can return the default value and set the Duration to something like 1ms, and disable fail-safe.

I think I'm missing something though, can you help me understand it more?

Thanks!

[jodydonetti]

Hi @marafiq

Application A sets a key value but never tries to get it. Instead, application B gets and removes the cache value and also would like to use the memory cache as the key will be accessed multiple times. Since the fusion cache allows the backplane notifications, when the cache value is changed by Application A, then Application B will refresh its value in the memory cache.

In other words, one process sets the value, and another process gets and removes the cache value.

Ok, this is quite clear.

A flag SkipMemoryCache on entry options like SkipDistributedCache so the memory cache can be skipped while setting it. It will be set to true in another process, thus using the memory cache.

I'm on it, will update you soon 🙂

[jodydonetti]

Hi @CM2Walki

for IFusionCache.SetAsync but the worker executing the TimerTrigger always ends up with the same 6GB of memory usage, whilst the workers only serving directly from the distributed cache are at less than 1GB.

I agree, this is strange.

I would also propose a SkipMemoryCache entry flag or to find out the reason why setting an instant MemoryCache expire doesn't free the underlying memory.

I'll investigate it, will let you know. Maybe I'll special-case it like with infinity.

This is related to #59, as Azure Functions when not self-hosted have a memory limit of ~1.5GB, being able to fine-tune the usages of the MemoryCache is important. You get billed for cpu and memory time.

ps: btw, my "fear" regarding totally disabling the memory cache is that, if you then call GetOrSet(), you may be convinced you are still protected from Cache Stampede & similar problems, but that is basically impossible without a memory cache.

[celluj34]

One question though: is fail-safe enabled in this case? I'm asking because even by handling TimeSpan.Zero natively, I cannot avoid saving it in the cache if fail-safe is enabled, because it would actually mean something like "save it as already expired, but keep it available for future problems as a fallback".

In our use case, fail-safes are not enabled. Having special handling for TimeSpan.Zero would work for me!

The problem I see is that GetOrSet() MUST return a value, so in the case of not wanting to return anything, what should FusionCache return?

Yes, this is a difficult problem, one that I had tried not thinking about, lol. Perhaps null? But I know there is a difference between returning a null value (which is a valid use-case) and not having a value to return.

In our case, what we're doing is returning a FluentValidation-style result. If it's valid, use it, otherwise return some error back up through the API. We had to do the try/catch and wrapper object initially because we were seeing strange problems when exceptions were thrown inside IMemoryCache.GetOrSetAsync using a custom "atomic" extension. The tl;dr is that we sometimes saw the value factory code run more than once when an exception was thrown, so this is our workaround which we've just carried forward (and may not even be neccessary anymore, but here we are).

[jodydonetti]

Hi all, this is the official issue to track the development of the feature.

The feature is basically done, with tests and everything, and I'll release it in the week end.

[jodydonetti]

Hi @CM2Walki

I've tried setting:

options.Duration = TimeSpan.Zero;
options.IsFailSafeEnabled = false;
options.DistributedCacheDuration = TimeSpan.FromDays(1);

for IFusionCache.SetAsync but the worker executing the TimerTrigger always ends up with the same 6GB of memory usage, whilst the workers only serving directly from the distributed cache are at less than 1GB.

See here for the special handling of "zero" durations 🎉

[jodydonetti]

Hi @celluj34

In our use case, fail-safes are not enabled. Having special handling for TimeSpan.Zero would work for me!

See here for the special handling of "zero" durations 🎉

[jodydonetti]

Hi all, v0.22.0 has been released and this is included 🎉