[FEATURE] Invalidation in future?

gercobrandwijk commented 2 months ago

Problem

When having a lot of situations that can invalidate the same cache entry, this means that a lot of factory executions are needed, which can cause a huge load on the database.

Solution

I would like to be able to invalidate the cache 'in the future'. For example you want to invalidate the cache in 5 minutes. In this 5 minutes window, when other requests/services are invalidating the same key, it will already be scheduled for over 5 minutes. This means that at max every 5 minutes the factory will be called onces.

Is this an already existing feature which I overlooked?

Alternatives

Any workaround is also fine for me :)

jodydonetti commented 2 months ago

Hi @gercobrandwijk , sorry but I don't understand the issue here.

The factory is involved/specified with GetOrSet calls, where you normally do something like this:

cache.GetOrSet<Product>(
    "my-cache-key",
    _ => LoadValueFromDb(), // THIS IS THE FACTORY
    TimeSpan.FromMinutes(5) // THIS IS THE DURATION
);

By doing this, you are guaranteed that only after 5 min the factory will be called again, and only once concurrently because of Cache Stampede protection.

If you are currently invalidating the cache entry via cache.Remove("my-cache-key") or cache.Expire("my-cache-key") calls, they will expire the cache entry sooner, so of course the factory will be called again at the first request after the Remove/Expire calls.

But if you want to avoid that and want the expiration to occur only once every 5 min, just don't call Remove/Expire manually and let the cache entry expire every 5 min automatically.

But maybe I'm missing something here, can you tell me more?

gercobrandwijk commented 2 months ago

I was in a hurry a bit, so I will try to make it a bit more clear...

Setting the duration to 5 minutes is not want we want, because that means in our scenario that the factory will run every 5 minutes, which is inefficient as well when looking at the whole picture.

Let me explain our scenario a bit more:

Cache key should be valid for 24 hours
On a certain moment of the day, a burst of requests is coming up from an external party (like 5000 requests), where every requests will have to invalidate the cache (we are talking about a daily user sync here).
The whole day (so also when the burst of requests are coming in) the cached item is widely used in our application

This means:

When the burst of requests is coming in, the cache key is invalidated a lot, while it also still being used by the rest of the application, this leads to a very high amount of factory executions
The rest of the day, almost never the cache key is invalidated

When that burst of requests is coming in (we do not know how many will come and what the latest request will be), I basically want to say 'invalidate this cache key over 10 min'. After that time period the burst of requests will be over, so than the factory will be called once (after it has been invalidated). Of course that means in that time period where the burst of requests comes in, the cached item has outdated information, but that's not a problem in our case.

jodydonetti commented 2 months ago

I was in a hurry a bit, so I will try to make it a bit more clear...

Setting the duration to 5 minutes is not want we want, because that means in our scenario that the factory will run every 5 minutes, which is inefficient as well when looking at the whole picture.

Let me explain our scenario a bit more: [...]

When the burst of requests is coming in, the cache key is invalidated a lot, while it also still being used by the rest of the application, this leads to a very high amount of factory executions

The rest of the day, almost never the cache key is invalidated

Ok now I understand, but just to be clear: we are talking about the very same cache key being invalidated repeatedly, right? Or different cache keys in sequence?

Anyway this seems like a very specific scenario, one for which I'm not sure a specific FusionCache feture would be the best way to go, since there are different considerations and edge cases to consider. For example, what should happen if you say "expire in 10 min" and 10 sec later somebody says "expire" (meaning a normal expire, meanng "now")? Should it expire now and cancel the one in 10 min or should it expire now AND after 10 min?

Because of your particular scenario, wouldn't it be enugh to start a timer in your own code that will expire the cache key after 10 min?

Let me know what you think.

whagoort commented 2 months ago

Hi, also joining the conversation here!

In my opinion it is indeed about the same key. And it if we would say "expire in 10 min", and afterwards somebody says "expire now" then it should just expire now.

Maybe we can also view this from the perspective that this would re-set the expiration to e.g. "10 min" if the expiration is longer then "10 min".

Setting a timer in other code is possible, but also not really easy in distributed scenario's, although that's not the most important argument.

gercobrandwijk commented 1 month ago

I was in a hurry a bit, so I will try to make it a bit more clear... Setting the duration to 5 minutes is not want we want, because that means in our scenario that the factory will run every 5 minutes, which is inefficient as well when looking at the whole picture. Let me explain our scenario a bit more: [...]

When the burst of requests is coming in, the cache key is invalidated a lot, while it also still being used by the rest of the application, this leads to a very high amount of factory executions

The rest of the day, almost never the cache key is invalidated

Ok now I understand, but just to be clear: we are talking about the very same cache key being invalidated repeatedly, right? Or different cache keys in sequence?

Very same cache key indeed.

Anyway this seems like a very specific scenario, one for which I'm not sure a specific FusionCache feture would be the best way to go, since there are different considerations and edge cases to consider. For example, what should happen if you say "expire in 10 min" and 10 sec later somebody says "expire" (meaning a normal expire, meanng "now")? Should it expire now and cancel the one in 10 min or should it expire now AND after 10 min?

It has to invalidate directly which means that the 'invalidate over 10 min' is superseeded by the 'now-invalidation', so that invalidation can be cancelled / will not be executed anymore.

gercobrandwijk commented 2 weeks ago

@jodydonetti Little bump.. What is your opinion/insight on this? What would be your advice to implement this in a reliable way which takes the concurrency/timing problems into account?

jodydonetti commented 2 weeks ago

Hi @gercobrandwijk , sorry for the delay but I just came back from my (very late) summer vacations 😅

What is your opinion/insight on this? What would be your advice to implement this in a reliable way which takes the concurrency/timing problems into account?

I don't feel like this is a candidate for an official feature for FusionCache, at least not right now, since the very specific edge case and potential consequences that have yet to be explored in full. Having said that, I think I can come up with something that is built on top of the core features: I'll try to make a POC and will update as soon as possible.

Thanks!

gercobrandwijk commented 1 week ago

Looking forward to your POC :)

gercobrandwijk commented 1 week ago

I tried to use the FusionCacheEntryOptions overload of the ExpireAsync, to let it expire in 15 seconds:

await _fusionCache.ExpireAsync(cacheKey, new FusionCacheEntryOptions() { Duration = new TimeSpan(0, 0, 15) });

But this seems not to be working; the factory is called directly after the Expire, instead of after 15 seconds.

Maybe this can be an addition to FusionCache (which will also help in my scenario)?

Thanks for all your efforts; love to hear for you! :)

ZiggyCreatures / FusionCache