support write/read splitting

[siddontang]

Now ledisdb slaves have no other effect but only replication for data security, we can let them serve some read operations to reduce master pressure if we don't care data consistency too much.

What do you think about this feature? @kouhate

[kouhate]

It definitely needed, and we have to determine how to set ReadPreference.

Then, I came up with the following way.

We can define some additional status for a slave like "standby" and "active", and provide a command to set or change <status> , e.g. codis-proxy server add <group_id> <redis-addr> <role> <status> or simply add <role> like "slave-standby" and "slave-active".

When the status is "standby", slave doesn't receive any access like before. On the other hand, after the status is changed to "active", slave accepts read commands.

However, this affects only per a group, so it is not so suitable for the people who really want to control a specific key routing. This would be a problem if there is a key which must be read from master, but read preference is set to read from slave.

So, maybe we should support some options or commands for client side to guarantee its routing. e.g.1) set like SELECT.

SETRP master      // read only from master.
GET abc

e.g.2) set as option.

 
GET abc WITHRP slave    // read only from slave

Is it possible and what do you think? @siddontang

[siddontang]

Hi @kouhate , maybe we can invite codis author @ngaut to discuss here too.

I prefer customizing commands, e.g, for every read command, we can add "FROM SLAVE" or "FROM MASTER" in the end.

I think "standby" may be unnecessary, if I add a redis into the replication topology, I want it be useful for something, not only for fault tolerance.

Btw, we may even do this: for a client connection, if it writes data for a key before, and we will let this client read this key data from master for a while (maybe 1s or 5s), then from slave.

[ngaut]

That's a cool feature. Command like "FROM SLAVE" will make it more clear for users. But it's not that easy to use. I guess they will like to using configuration to do it transparently.

[siddontang]

Hi @ngaut , codis will use this feature later?

[ngaut]

@siddontang Do not have any plan right now. I prefer to using scale to get better performance. Any way i like consistency :)

[kouhate]

@siddontang, that's cool to use "FROM SLAVE". For easiness, it is also nice that a key written just now always routes to master.

"standby" is my fault, not necessary for simple use case:-) Instead of this, why don't you support "balance" for every nodes like this?

              balance
master        2
slave1        2
slave2        1

This means master get read operation with a 2/5 possibility, slave1 get 2/5, slave2 get 1/5. If slave3 is added just now, its balance may be started from 1, but perhaps its I/O load may be too heavy to perform efficiently at this time. So, something to start gracefully like balance 0 is needed in this case. And someone want to add slave4 only for the purpose of near realtime backup using a cheap machine.

I think it's good to set those feature on server side globally, then users can modify its feature on client side if needed.

But, If you want to avoid complexity, the idea you said before is the best.

[siddontang]

Hi @kouhate , supporting weight for different server may be interesting, but I think I have no time to do it. :disappointed:

These days, I have been studying redis cluster, elasticsearch and other distributed NoSQLs and think maybe we can do better. I will discuss with you using google hangout later.

[kouhate]

OK @siddontang, I hope you could make it good.

Btw, that idea is came from a certain NoSQL already used in production.

[siddontang]

Hi @kouhate, Sorry that I mislead you, what I said to do better is not for balance or other xcodis, but that maybe we can do a better cluster solution like elasticsearch supporting node found automatically, data resharding automatically, etc.

So I said if I have any idea, I will contact you with google hangout, not here.

[kouhate]

Aha, I don't expect you implement the balance feature. Let me talk on hangout about those optional cluster feature.

Then, is it ok for you to support FROM SLAVE? I suppose FROM MASTER or FROM SLAVE are still useful for most of users even if you support the feature which read from master for a while and then from slave automatically. @siddontang

[siddontang]

Yes, I will try to support FROM MASTER, FROM SLAVE and TEMPSELECT db later. :smile:

[kouhate]

Cool!