MichaCo / CacheManager

CacheManager is an open source caching abstraction layer for .NET written in C#. It supports various cache providers and implements many advanced features.
http://cachemanager.michaco.net
Apache License 2.0
2.34k stars 457 forks source link

Query: To add & get multiple key values. #239

Closed girishnandaniya closed 6 years ago

girishnandaniya commented 6 years ago

Hello @MichaCo - We are going to implement caching in our application. As per our requirement, your library is most suitable for us. We are planning to use Redis & System runtime caching. This library has most of the required functionality but we require some more methods. Like

We shaw the flow of Add a single key and Get a single value.

We would like to customize this library based on our requirement so can you please suggest a way to achieve this? We are thinking to follow the same flow as a single key.

Below are the quires regarding the add & get a single key.

  1. What is the purpose of trigger onGet?
  2. How can we add multiple keys using Lua script or without Lau script?

Or suggest any better alternative way if you have.

MichaCo commented 6 years ago

You'll have to run multiple gets or adds within a simple loop. There is no way to optimize that further. Depending on your scenario, you can use Put instead of Add which runs in fire and forget for the Redis implementation and will be much faster than a Add. But that's about it.

That's pretty much why those methods do (and will not) exist in this library.

girishnandaniya commented 6 years ago

@MichaCo Thanks for the response.

First, we tried to add multiple keys with below Lua script and it works.

For example "eval "return redis.call('HMSET', 'myhash','Key1','value1','Key2','value2')" 0"

We tried to do same with Lua script in the library but we have observed that from the Lua put a script in the library there is no hash defined. In above example "myhash" is the hash.

We are trying to update script to insert multiple keys. Would you please help us if it possible?

Here is the Lau put script from the library. local result=redis.call('HMSET', KEYS[1], 'value', ARGV[1], 'type', ARGV[2], 'expiration', ARGV[3], 'timeout', ARGV[4], 'created', ARGV[5], 'defaultExpiration', ARGV[6]) redis.call('HINCRBY', KEYS[1], 'version', 1) if ARGV[3] > '1' and ARGV[4] ~= '0' then redis.call('PEXPIRE', KEYS[1], ARGV[4]) else redis.call('PERSIST', KEYS[1]) end return result

MichaCo commented 6 years ago

You are using HMSET to set multiple fields of a hash. That's not equal to setting multiple keys in Redis.

girishnandaniya commented 6 years ago

@MichaCo Thaks for your prompt reply. What we are trying to achieve it can be achieved with MSET. I guess where we don't require hash.

girishnandaniya commented 6 years ago

@MichaCo - We performed load testing on PUT & GET methods of the library using for loop. I have attached the statistics of the load test. cachemanagerperformance

It seems that GET & PUT is not performing well with the loop only for Redis. It would be better if we can add support for multiple PUT & GET.

MichaCo commented 6 years ago

Not sure what exactly you are doing in code. But with a simple test, inserting 100000 strings takes <1sec, reading them one by one takes ~5-6 seconds.

Reading from memory (with in-memory layer and all keys in memory) is obviously faster,

private static void Main(string[] args)
{
    var cache = CacheFactory.Build<string>(
        c => c
        .WithDictionaryHandle().And
        .WithRedisConfiguration("redis", "localhost,allowAdmin=true")
        .WithJsonSerializer()
        .WithRedisCacheHandle("redis"));

    cache.Clear();

    var swatch = Stopwatch.StartNew();
    var keys = new List<string>();

    for (var i = 0; i < 100000; i++)
    {
        var k = Guid.NewGuid().ToString();
        keys.Add(k);
        cache.Put(k, k);
    }

    Console.WriteLine("Elapsed Put: " + swatch.ElapsedMilliseconds);
    swatch.Restart();

    foreach (var key in keys)
    {
        var x = cache.Get(key);
        if (x == null || x != key)
        {
            throw new Exception();
        }
    }

    Console.WriteLine("Elapsed Get: " + swatch.ElapsedMilliseconds);
}

Prints

Elapsed Put: 465
Elapsed Get: 40