support for hiredis pipelining

[saveriocastellano]

first of all thanks for this great project.

I was wondering if you have wrapped the redisAppendCommand(Argv) and redisGetReply function from hiredis that allow one to do pipelining..

thanks

[shinberg]

Hi saveriocastellano. Thank you for feedback. Redis cluster doesn't support pipelining out from the box since client have no idea about to wich cluster nodes requests would send to. So client can do pipelining to different keys to different nodes, so that's wouldn't working.

[saveriocastellano]

Thanks for your reply. Actually even with redis cluster it does make sense to do pipelining when you have to send many redis commands altogether and the underlying keys of all those commands are all mapped to the same hash slots.

I have modified your library to support pipelining, and I think I got it working. There are two remarks for the modified code to work:

1) all keys involved in the pipeline must be mapped to the same hash slot so that it is guaranteed that the commands will be executed by the same node (otherwise it doesn't even make much sense to use pipeline)

2) the pipeline must be started with an additional redis command that is used to determined/set the connection to the proper redis node that will execute all commands in the pipeline

So to use the pipeline let's say you want want to set the following keys:

{keyslot}key1 {keyslot}key2 {keyslot}key3 {keyslot}key4 {keyslot}key5

I'm using {keyslot} as part of the keys because redis guarantees that when key contain portions enclosed in curly brackets then only the content of the curly brakets will be used to calculate the hash slot and therefore in this case it is guaranteed that all keys above will be handled by the same node of the cluster.

So here are the commands I added in the code I modified, and here is how to use them:

1) you call 'startpipeline' using the part of the key included in curly brackets (in this case "keyslot")

RedisCluster::HiredisCommand<>::StartPipeline( ctx->conn, key);

2 you append all other commands to the pipeline:

RedisCluster::HiredisCommand<>::AppendCommand(ctx->conn, cmd);

3) you call 'getReply' to get the reply for each command:

redisReply *reply = RedisCluster::HiredisCommand<>::GetReply();

4) you call 'endpipeline' to end/close the pipeline

RedisCluster::HiredisCommand<>::EndPipeline();

The above code seems to work good. However I have a question....

In your library when you calculate the hash slots from a key, do you act exactly like redis does internally? I mean, if the key contains a portion enclosed in curly brackets do you only consider the part within the curly brackets to calculate the hash slot? From my understanding this is what Redis does internally, so it is crucial for my modified code to work that your library also does the same.

[saveriocastellano]

Here my modified file in case you want to take a look at the code I added:

hirediscommand.zip

[shinberg]

Ok, i understand, i'll try to make it.

Answering your question about calculating hash slot - so yes it is made like redis cluster mades it for itself. This code is posted in some of official documentation pages of redis.

[allthetime]

@saveriocastellano are you still using this modified code? I am in need of pipelining support and curious if anything has developed since this thread was created.