StackExchange / NRediSearch

Other
31 stars 9 forks source link

no batch processing for redis search #9

Open kishoretvk opened 5 years ago

kishoretvk commented 5 years ago

there is no support for redis search batch add hashes, we have to call the add function in async. which is not super fast, for adding a million or 10 million records

mgravell commented 5 years ago

can you be more specific? is there a particular command you would have this execute? note that NRediSearch was a port of JRediSearch - is there a JRediSearch API this should mimic? Basically: what would you have this do, in terms of redis behavior?

kishoretvk commented 5 years ago

Thanks for responding, here are some more details. in normal redis client we create a batch for hashes or sets and push them all at once. example code

var redis = RedisStore.RedisCache;          

            RedisKey alphaKey = "alphaKey";
            RedisKey betaKey = "betaKey";

            //Batching
            var list = new List<Task<bool>>();
            var keys = new List<RedisKey> {alphaKey, betaKey};
            IBatch batch = redis.CreateBatch();

            //add the delete into batch
            batch.KeyDeleteAsync(alphaKey);

            foreach (var key in keys)
            {
                var task = batch.StringSetAsync(key, "123");
                list.Add(task);
            }

            batch.Execute();

            Task.WhenAll(list.ToArray());

however in NredisSearch, we do not see such batch, we create an index

var index = "TestIndex";
var tasks = new  List<Task<bool>>();
var clientsearch = new Client(index, rediscache);
foreach(hashdata in hashresults)
{
var dataresult = clientsearch addhashasync(hashdata .id, hashdata.data , nosave);
taks.add(dataresult);
}
Tasks.Whenall(tasks.ToArray());

is there a way in which we can add batch support here as well?