add a fork/join commands in protocol

[tomcucinotta]

This is useful to model replication protocols: a node would receive a message containing a special FORK message that requests forwarding a subsequence of commands that follow in the same request to multiple node servers; those servers would simply execute processing and/or load/store workload as required, then execute a JOIN to reply back to the forking node; the forking node would be waiting for either all or a quorum of the replies, then execute a REPLY back to the client.

So, the workflow could be something like:

Request 1 COMPUTE(time) 2 FORK(dest1:port1, dest2:port2, dest3:port3; num_cmds=3) 3 COMPUTE(time) 4 STORE(bytes, sync=t/f) 5 REPLY 6 JOIN(type=all/quorum) 7 REPLY

The forking node would only forward to the "slaves" the 3 commands from 3 to 5, then wait for the REPLYies to come in the JOIN command, then REPLY to the client.

FORK might also be realized as an asynchronous variant of the current FORWARD cmd, i.e.: 1 COMPUTE(time) 2 FORWARD(dest1:port1, async=t, cmds=5..7) 3 FORWARD(dest2:port2, async=t, cmds=5..7) 4 FORWARD(dest3:port3, async=t, cmds=5..7, skip=3) 5 COMPUTE(time) 6 STORE(bytes, sync=t/f) 7 REPLY 8 JOIN(type=all/quorum) 9 REPLY

Here, the forking node would forward asynchronously commands from 5 to 7 only, then skip them (only in the last FORWARD) so to jump to the JOIN command, then REPLY to the client.

As from both examples, the JOIN command might have a type option, set to all if we need to wait for all the replies, before continuing, or quorum if we need to wait for just a majority of them. Similarly to the MongoDB r/w "concern", this might actually be an integer telling the JOIN command how many of the replies are expected, before moving forward.

[deRemo]

Tackling the issue client-side

[tomcucinotta]

addressed by MULTI_FORWARD in 4aa467f3, closing