Load Balancing Across Routers

[mbonneau]

This is in response to https://github.com/crossbario/crossbar/issues/221

[mbonneau]

Hi @oanhnn ,

First of all, I don't have experience running a router with 100,000 concurrent users. The most I have tested is a few thousand.

The ability of Thruway to handle that many connects could probably be answered best by @cboden , the maintainer of the Ratchet project (as Thruway uses Ratchet under the hood). In Thruway, we stay away from anything that would block, so I would imagine we are not adding significant overhead.

As Thruway does not support multi-router configurations, the first thing I would try is to do is improve performance enough to allow a single router to handle all your traffic (if that is possible with Ratchet).

I would make sure that you have performance optimized. Take look at http://socketo.me/docs/deploy from the Ratchet project. There is a lot of good information about deployment, and with lots of connections, you should make sure you are running with libevent (if you are not already).

It sounds like you are using "internal clients". If this is the case, you could remove the client from the router and significantly improve performance (especially if you are doing any I/O).

While Thruway doesn't support multi-router configurations right now, a single router can load balance RPC calls. So if the limitations of your system are caused by blocking calls to databases or other blocking I/O on the "server" side, you can register many "server" clients to handle these requests, even across multiple machines.

Getting routers to work together will take a little extra work. Can you tell me a little more about the way your applications works with the publish/subscribe and RPCs?

[oanhnn]

Thank @mbonneau for support.

My system is backend for a fighting game (one player vs on player real-time) and chat system. I use multi-router and load balancer to load sharing and backup router (very important). When user (client) connected, system will auto matching this user with an available user (user in matching list) . If not other user, this user is put to matching list and subscribe topic matching.success. This user is receive publish message when has matching success with other user. And when user playing, user call RPC. I use internal client for game logic, chat (log chat messahe to DB).

Can you suggest me some solutions? I am used Redis between Internal client and DB. I am used libevent and ext-event

[pronskiy]

Hello @mbonneau and @oanhnn,

I'm implementing chat in a distributed system. Basically have the same requirement - get routers to work together. I appreciate any help on this topic. Maybe suggestions and ideas on how to implement it better?

[oanhnn]

Dear @pronskiy I have not solution and current i used only one router. Important problem is we haven't backup router when error. @@ I think you can use Redis (other shared memory engine) to share data between your routers.

[mbonneau]

Hi @oanhnn and @pronskiy ,

I have been thinking about solutions to these use-cases of Thruway. I do have some ideas to try and will get back to you when I have a chance to test them out.

[pronskiy]

Hi @mbonneau,

Did you have time by any chance to think about this one? If not, maybe you could share your ideas with me, and I'll try to implement them. What do you think? Appreciate your help.

[pronskiy]

Hi @mbonneau,

Please, let me know if I can help somehow.

[pronskiy]

Hello @mbonneau,

I'm still trying to implement some PoC for this. As @oanhnn suggested it is possible to use Redis and it's pub/sub to pass events between nodes. For example socket.io implements such approach http://socket.io/docs/using-multiple-nodes/

But due to certain limitations on customers' setups, I have to consider solutions without more dependencies. Maybe keeping additional connection between nodes.

Anyway, I would be glad to hear advice from you on how to better extend Thruway to do that. Currently I'm thinking of implementing RouterModule/RealmModule and somehow hook PublishMessageEvent event. Does this way has chances? What would you recommend better?

Thank you!

[mbonneau]

Hi @pronskiy ,

First of all, the 0.4/dev-master branches are still experimental, so things may change in the future.

Hooking the event might be a good way to go on your chat system.

One thing that may or may not be an issue for you is that the session that is publishing will not exist on the other router.

As for RPC's, if you are only using them on the "backend" of the system, you could run multiple backend clients or design your clients to connect to multiple routers.

I still have not been able to come up with a great way to make routers transparently "cluster" or load balance. Most solutions will need to be tailored to the application. For instance, @oanhnn 's needs may not be met by this idea that we are discussing. If he is publishing many messages per second for each frontend client - the messages would all get forwarded to every other router, causing every router to process every message even though they are for only one recipient.

@oanhnn , in your situation, if you are pairing people together, it might be better to have a separate system for matching opponents (which could be setup similar to the above) and then instructing the clients to both connect to the same router to process the actual fight information.

[pronskiy]

@mbonneau thanks a lot for the response!

In my wamp server-side client a have a dict of user->sessionid items, so I would most likely depend on this table when sending messages over to other nodes, to make them not broadcasted but targeted.

Also, I've implemented WAMP support for this tool M6Web/websocket-bench for benchmarking. Was interesting how do you guys test load?

[RafaelKa]

Hi, not sure if my opinion interesting anyone but nevertheless:

I think the best way for balancing Thruway routers is Redis in combination with Predis\Async & RedisRPC.

Can somebody show the point in source, where Thruway stores subscriptions and delegates publishing to subscribers?

[mbonneau]

This is outside of the current scope and would likely be solved by the library user at this point. I am closing this as it has been inactive for years.