search

shinberg / cpp-hiredis-cluster

c++ cluster wrapper for hiredis with async and unix sockets features
BSD 3-Clause "New" or "Revised" License
65 stars 26 forks source link

Find keyslot for key and override usual calculation for a command by telling it which node to connect to #26

Open allthetime opened 6 years ago

allthetime commented 6 years ago

Hi, I have a use case where I send many commands in a short period of time with the same keyslot.

I did some performance profiling (gprof) and it looks like slot-finding related functions are quite expensive.

What I would like to do is find the node for a give key once and then run a bunch of commands with that knowledge already.

Is this possible with this library?

If not, do you know what function is responsible for selecting the node from a given key? Then I could just use that one time and selectively send all the messages to the proper redis node.

shinberg commented 6 years ago

Hi, Thanks for report, but I honestly don't understand how you found out that the keyslot finding functions are expensive, but at the same time you cannot understand how that functions are called?

The keyslot finding structures are related to 

        typedef std::map <SlotRange, redisConnection*, typename RCluster::SlotComparator> ClusterNodes;
        typedef std::map <Host, redisConnection*> RedirectConnections;

and the following method: inline static typename Storage::iterator searchBySlots( typename RCluster::SlotIndex index, Storage &storage ) so that for finding the keyslot they are used. But they are unlikely to be expensive unless you have more than say 100 instances in your hiredis cluster. But I think you should have even more that the noticed amount for keyslot finding operation to be slow.

Anyway you can define your own node-search class (see class DefaultContainer in cpp-hiredis-cluster/include/container.h) and pass it as template argument to all the method invocation like in the following example:

typedef Cluster<redisContext, MyConnectionContainer<redisContext> > ThreadPoolCluster;
cluster_p = HiredisCommand<MyConnectionContainer>::createCluster( "127.0.0.1", 7000 );
reply = static_cast<redisReply*>( HiredisCommand<MyConnectionContainer>::Command( cluster_p, "FOO", "SET %s %s", "FOO", "BAR1" ) );

For more information you can follow the example of thread-safety connection container in the src/examples/threadpool.cpp path of this project.