[QUESTION] How can I trigger "eager refresh" from a service that reads the cache, but can't produce the value

[zorgoz]

ref: https://stackoverflow.com/questions/79130868/triggering-eager-refresh-with-fusioncache-from-consumer-side

Problem

I have a service that can produce some data. And some others that are consuming that data. The consumers can request the data over masstransit. But as producing that data is relatively expensive and prone to third party service outages, I intend to use FusionCache with FailSafe and EagerRefresh. The consumers are on the same Redis backplane with the producer, hence, it is straightforward for them to try to get it from the cache first and only address the producer service if it is not found there. I use .TryGetAsync() for that. (In some cases the consumers need to bypass any cache so should the producer, but that's irrelevant here.)

If I got it correctly, to trigger the eager refresh - as only the producer can produce the data -, the producer needs to try to get the value from the cache in the grace period.

I don't know if I can make the TryGet believe (in the same grace period) the value is expired so that I can address the service and trigger the eager refresh. If I understood correctly, the getter is not that sophisticated, only GetOrSet will consider the EagerResfresh ratio.

Workaround

• One workaround that could work is to read the entry from memory cache as it is, with metadata and all as a second step if the MaybeValue has the value. But FusionCacheMemoryEntry<TValue> is internal, although could be replicated to better match this purpose.

Solution

• The simplest non-breaking change would be to have the metadata in MaybeValue as a prepopulated property to read or an a .GetMetadata() if prepopulating it would be too expensive.

[zorgoz]

Supposing I have:

.AddMemoryCache()
.AddFusionCache()
.WithRegisteredMemoryCache()

this is the implementation for my workaround (fortunately FusionCacheEntryMetadata is public):

    bool TryGetMetadataFromMemory<TValue>(string key, out FusionCacheEntryMetadata? metadata)
    {
        metadata = null;
        if(!memoryCache.TryGetValue<object>(key, out var value) || value is null) return false;

        var prop = value.GetType().GetProperty("Metadata", typeof(FusionCacheEntryMetadata));

        if (prop is null) return false;

        metadata = (FusionCacheEntryMetadata)prop.GetValue(value)!;

        return true;
    }

(PS: the step-by-step guide seems to be missing the .WithRegisteredMemoryCache() there. I had nothing in memory after fetching from the distributed cache without it.)

[jodydonetti]

Hi @zorgoz

Problem

I have a service that can produce some data. And some others that are consuming that data. The consumers can request the data over masstransit. But as producing that data is relatively expensive and prone to third party service outages, I intend to use FusionCache with FailSafe and EagerRefresh.

I'd say good idea!

The consumers are on the same Redis backplane with the producer, hence, it is straightforward for them to try to get it from the cache first and only address the producer service if it is not found there. I use .TryGetAsync() for that.

Red flag here: the only way to be protected from Cache Stampede is to use a GetOrSet call: by not using it and opting for separate GET + SET calls (eg: a TryGet(...) followed by production of the value followed by a Set(...)) you are not allowing FusionCache to coordinate the calls and prevent multiple factories to run for the same cache key (eg: what is known as request coalescing, see the link above).

So I would simply use GetOrSet(...) and in the factory (if I got it right) ask for the data via MassTransit.

If I got it correctly, to trigger the eager refresh - as only the producer can produce the data -, the producer needs to try to get the value from the cache in the grace period.

Wait, I think I'm missing something here: above you stated that "The consumers can request the data over masstransit", but now you are saying that only the producer can produce the data. Maybe a consumer can ask the producer to produce via MassTransit, but in a kind of fire-and-forget way? And then the production will happen on the producer side which in turn will update the cache?

Can you clarify it?

Anyway, let's say it's how I imagined it above:

• a consumer can ask via MassTransit to a producer to produce it, but cannot wait for the result
• a producer can be contacted via MassTransit, generate the data and update the cache

If so, I can suggest an idea you can play with: you can use some of the methods normally used for Conditional Refresh like ctx.NotModified(), and maybe a touch of Adaptive Caching too, in this way:

// CONSUMER SIDE
var product = await cache.GetOrSetAsync<Product>(
    $"product:{id}",
    async (ctx, ct) =>
    {
        // CALL MASS TRANSIT HERE...

        if (ctx.HasStaleValue) {
            // TEMPORARILY RENEW THE OLD VALUE
            ctx.Options.Duration = some_duration_here;
            return ctx.NotModified();
        }

        // NO STALE VALUE -> PRODUCER DID NOT PRODUCED THIS YET
        ctx.Options.Duration = maybe_some_different_duration_here;
        return some_default_value;
    },
    opt => opt.SetDuration(duration).SetFailSafe(true)
);

// PRODUCER SIDE
// 
public async Task Consume(ConsumeContext<CreateData> context)
{
    // HEAVY COMPUTING HERE...
    await _cache.SetAsync("cache-key", myData);
}

Could this work?

[jodydonetti]

(PS: the step-by-step guide seems to be missing the .WithRegisteredMemoryCache() there.

Because normally it's not needed, since FusionCache will create its own private MemoryCache instance, not shared with others. In general, sharing a MemoryCache can give you problems because collisions, conflicting configurations (eg: SizeLimit), etc.

I had nothing in memory after fetching from the distributed cache without it.

This... I don't understand.

[zorgoz]

Hello,

I hear you, but hear me out about my concern: the stampede protection works only on one consumer node. Or more precisely for each node individually. While I do wait for the producer on one consumer node, a sibling node still can get hit on the sibling endpoint. And it won't find anything in the cache either (let's assume this scenario). As a consequence, this other node will also reach out to the producer. Hence, in my understanding, I need protection on the producer side. Not only that, but all consumers would write to the distributed cache the same stuff, and I want to prevent that. That's why I use only the getter only the consumer side, and have another kind stampede protection on the producer side.

What I would need is something similar to the flags in HybridCache where I can instruct the cache per call if should read/write to the distributed/memory cache. I know I can omit the distributed cache altogether in your library (FusionCacheEntryOptions.SkipDistributedCache), but I would need to be able to decide separately for read and write. With denying write to the distributed cache on the consumer side altogether, and allowing reads only when in a non-forced scenario, I could provide this inside a GetOrSet on the consumer-side. But as I see now, my only option is to let multiple 'gets' slide in, and if, by any chance they are concurrent, the producer-side protection will still kick in.

[jodydonetti]

I hear you, but hear me out about my concern: the stampede protection works only on one consumer node. Or more precisely for each node individually. While I do wait for the producer on one consumer node, a sibling node still can get hit on the sibling endpoint. And it won't find anything in the cache either (let's assume this scenario). As a consequence, this other node will also reach out to the producer.

Correct, but can't this be handled on the producer side? What I mean is that by receiving a MassTransit call while another one is being processed, the second one would simply be ignored (or, depending on your needs, both at the same time or inside a certain time frame, like 1 min or similar).

For the "how", I'm thinking about something simple like using a SemaphoreSlim. Another way would be to simply use... FusionCache itself. The approach could be something like this: when the producer receives the MassTransit request, it will expire the cache entry and then immediately do a GetOrSet. The GetOrSet includes cache stampede protection, so even if 10 MassTransit requests come in at the same time, only the first one will be effectively executed.

Just spitballing an idea here, let me know what you think.

Hence, in my understanding, I need protection on the producer side. Not only that, but all consumers would write to the distributed cache the same stuff, and I want to prevent that.

In theory only the first consumer would write on L2, the others would find the updated data already there.

What I would need is something similar to the flags in HybridCache [...]

Yeah I'm thinking about adding SkipMemoryCacheRead, SkipMemoryCacheWrite, SkipDistributedCacheRead and SkipDistributedCacheWrite, and have the current one become "synthetic": what this means is that the getter will return true only if both the read and write are true, and the setter would set both to the specified value.

Thoughts?

[zorgoz]

Yes, I have something similar now: consumer:

• if not forced: try to get,
• if not found or forced address the producer

producer:

• if forced: produce and store (Set)
• if not forcer: GetOrSet (produce if needed)

My approach on producer-side is to make use of the multi-awaitability of the same task: simply put any concurrent request for the same set of data will get the same task in return while the production is is progress. When finished, by the time any new consumer would look out for the same data, it should have that already in the distributed cache.

Looking forwar for your Skip... implementations!

[jodydonetti]

HI @zorgoz , are there some missing points about this?

I'm asking so can I close this, for house keeping.

Thanks.

[zorgoz]

@jodydonetti Nope. I am looking forward to the features you promised. :D Thank you for your great work!