[BUG] Slow Performance of FusionCache compared to Microsoft.Extensions.Caching.Memory.MemoryCache

[raghumirajkar]

Describe the bug

Slow Performance observed in frequently used operations like Get/Set/Remove

Method	Mean	Error	StdDev	Gen0	Allocated
SetWithMemCache	132.77 ns	1.532 ns	1.358 ns	0.0253	320 B
GetWithMemCache	27.62 ns	0.458 ns	0.406 ns	-	-
RemoveWithMemCache	46.05 ns	0.447 ns	0.396 ns	-	-
SetWithFusionCache	250.11 ns	4.873 ns	6.336 ns	0.0296	376 B
GetOrDefaultFusionCache	50.30 ns	0.450 ns	0.376 ns	-	-
RemoveFusionCache	61.47 ns	0.641 ns	0.536 ns	-	-

To Reproduce

Here's a MRE (Minimal Reproducible Example) of the issue:

using BenchmarkDotNet.Attributes;
using Microsoft.Extensions.Caching.Memory;
using ZiggyCreatures.Caching.Fusion;

namespace MemoryAndFusionCacheBenchmark;

[MemoryDiagnoser]
public class MemAndFusionCache
{
    private MemoryCache memCache = new(new MemoryCacheOptions());
    private FusionCache fusionCache = new(new FusionCacheOptions
    {
        DefaultEntryOptions = new FusionCacheEntryOptions
        {
            Duration = TimeSpan.FromSeconds(1),
            Priority = CacheItemPriority.Low
        }
    });
    private string key1 = "key1";
    private string val1 = "val1";

    [Benchmark]
    public void SetWithMemCache()
    {
        memCache.Set(key1, val1,
            new MemoryCacheEntryOptions { AbsoluteExpirationRelativeToNow = TimeSpan.FromSeconds(1) });
    }

    [Benchmark]
    public void GetWithMemCache()
    {
        var val = memCache.Get(key1);
    }

    [Benchmark]
    public void RemoveWithMemCache()
    {
        memCache.Remove(key1);
    }

    [Benchmark]
    public void SetWithFusionCache()
    {
        fusionCache.Set(key1, val1);
    }

    [Benchmark]
    public void GetOrDefaultFusionCache()
    {
        var val = fusionCache.GetOrDefault(key1, string.Empty);
    }

    [Benchmark]
    public void RemoveFusionCache()
    {
        fusionCache.Remove(key1);
    }
}

Expected behavior

• expected fusion cache to be more performant

Versions

I've encountered this issue on:

• FusionCache version 1.0.0
• .NET version net6.0
• OS version Windows 10 Enterprise

Screenshots

If applicable, add screenshots to help explain your problem.

Additional context

• are we missing any additional settings/configuration?

[jodydonetti]

Hi @raghumirajkar and thanks for using FusionCache.

expected fusion cache to be more performant

FusionCache underneath uses IMemoryCache for L1 and IDistributedCache for L2, see the intro here:

It uses a memory cache (any impl of the standard IMemoryCache interface) as the primary backing store and, optionally, a distributed cache (any impl of the standard IDistributedCache interface) as a secondary backing store for better resilience and higher performance, for example in a multi-node scenario or to avoid the typical effects of a cold start (initial empty cache, maybe after a restart).

Because of this it cannot physically be faster than MemoryCache, since it is in fact MemoryCache + extra features.

One question you may pose yourself is "then why should I use FusionCache, instead of simply using MemoryCache?" and the answer to that are all the extra features of FusionCache like cache stampede prevention, an optional 2nd level, fail-safe, soft/hard timeouts, backplane, auto-recovery and many more.

Hope this helps, let me know.

PS: I'm experimenting with a new version that will use a more optimized memory cache for the L1 then MemoryCache, but there's nothing to see yet.

PPS: I'm also keeping an eye on this which may or may not come with .NET 9.

[madhub]

@jodydonetti Have you explored alternative like https://github.com/bitfaster/BitFaster.Caching for L1 cache instead of .NET MemoryCache ?

[jodydonetti]

Hi @madhub , yep totally, on top of others. It's something I'm experimenting with from some time, playing with different designs, benchmarking them, etc.

One idea may be to have a new IFusionCacheMemoryLevel or something like that so that we can switch between different implementations, like we already can with IDistributedCache or IFusionCacheBackplane.

Another idea is that I may also end up creating an optimized one specifically for FusionCache, and use that instead of depending on a 3rd party one.

Again, I'm playing with different designs and waiting to see what .NET 9 will bring to the table.

In the case of BitFaster.Caching for example there's the problem that the Duration is per-cache and not per-cache-entry, see here.

[jodydonetti]

I'm closing this: if you think I've missed something let me know and I'll gladly re-open it.