DRPC traffic, traffic-control and something else about design

[wsky]

1. app1 ----> traffic-server
2. traffic-server ----> app2
3. app2 ----> traffic-server
4. traffic-server ----> app1

buffer usage:

• at 1: buf1=pool.get()
• at 2: buf1.release()
• at 3: buf2=pool.get()
• at 4: buf2.release()

bufferPool should be a mem limit pool as server's protection.

if app1 not reliable, at 2, need design a traffic-control module, and fast-fail at 1.

using write timeout at 2,4

[wsky]

DRPC traffic

• G = Gateway, connection management, face to connection
• R = Route, client/connection route
• C = Client
• C1/2/3 = Connection
• C1/C1'/C1''= Connections of C
• Black = SEND message path
• Red = SEND_ACK message path

[wsky]

Message protocol

• JSON, used for all income message to G, easy to implement for client
• Binary, private protocol, used for internal messaging, performance first

id = income message id oid = outcome message id b =message body

FROM	TO	PROTOCOL	MSG	Resolve MSG(at TO)
SEND
C1	G1	JSON	{ 'id':1, 'to':'C2', 'b':{} }	true
G1	R1	Binary	[ CMD, { 'oid':1, 'b':{} }, ID, OID, TO, [ C1 ] ]	false
R1	G2	Binary	[ CMD, { 'oid':1, 'b':{} }, ID, OID, TO, [ C1, G1 ] ]	false
G2	C2	JSON	{ 'oid':1, 'b':{} }	true
SEND_ACK
C2	G2	JSON	{ 'oid': 1, 'b':{} }	true
G2	R1	Binary	[ CMD, { 'id': 1, 'b':{} }, ID, OID, TO, [ C1, G1 ] ]	false
R1	G1	Binary	[ CMD, { 'id': 1, 'b':{} }, ID, OID, TO, [ C1 ] ]	false
G1	C1	JSON	{ 'id': 1, 'b':{} }	true

message rebuild at G1

C1 => { 'id':1, 'to':'C2', 'b':{} } -> { 'oid':1, 'b':{} } => C2

message rebuild at G2

C2 => { 'oid': 1, 'b':{} } -> { 'id': 1, 'b':{} } => C1

Retained message at G2:

[ CMD, { 'oid':1, 'b':{} }, ID, OID, TO, [ C1, G1, R1 ] ]

Forward

[ C1, G1, R1 ]

 * +-----+-----+-----+------------+--------------+----------+-------------+----------+------+
 * | CMD | Len | Dst | Header-Len |   Headers    | Path-Len |    Path     | Body-Len | Body |
 * |     |     |     |            |--------------|          |-------------|          |      |
 * |     |     |     |            | Type | Value |          | Flag | Path |          |      |
 * +-----+-----+-----+------------+------+-------+----------+------+------+----------+------+
 * |  1  |  4  |  8  |     1      |   1  |   N   |    1     |   1  |   8  |     4    |  N   |
 * +-----+-----+-----+------------+------+-------+----------+------+------+----------+------+
 * | 0/1 | int | long|    byte    | 1/2/ |       |   byte   |  0/1 | long |    int   |      |
 * +-----+-----+-----+------------+------+-------+----------+------+------+----------+------+

[wsky]

Messaging workflow

Public

Resolved in Gateway, client knows

• SEND
• [ MSG_ID, TO ]
• SEND_ACK
• [ OUT_ID ]
• ONE_WAY
• [ ]
• MESSAGE
• MESSAGE_ACK

Internal

Traffic between internal nodes

• FORWARD
• [ C1, G1, R1, G2, C2 ]
• BATCH
• [ [ C1, G1, R1, G2, C2 ], [ C1, G1, R1, G2, C2 ], ... ]

[wsky]

Identity and forward

• Node: G, R
• Client: C
• Connection: C1,C1', C1''

node foward: [G1, R1, G2, ...] forward to connections [C1, [G1, R1, G2, ...]]

[wsky]

Node traffic

N is local or remote node, R is router,

route info sync between all routers, DRPC traffic build on top of node traffic

var r = new RouteNode(meta);
var c = new ConnectionNode(connection);
r.register(c);

c.onMessage(function(msg){
    forward(msg);
});
r.onMessage(function(msg){
    forward(msg);
});

function forward(msg){
   var next = msg.getNext();
    // ... 
}

• Router:Node
• Connection:Node
• Connector

[wsky]

Expose

• public
  • client
• internal
  • route