tomcucinotta / distwalk

Distributed processing emulation tool
GNU General Public License v3.0
1 stars 4 forks source link

FORK/JOIN with arbitrary branch contents #41

Open tomcucinotta opened 6 months ago

tomcucinotta commented 6 months ago

Currently, the MULTI_FORWARD allows us to emulate topologies where parallel branches are an exact replication of the same commands. Indeed, each IP is forwarded the same command-sequence excerpt from the original request containing the MULTI_FORWARD. We might add a more general mechanism allowing for arbitrary FORK points, where individual parallel branches may be instructed to execute different command sets, and JOIN points, where these individual asynchronous branches re-synchronize. In terms of behavior, a FORK as just mentioned might resemble a FORWARD in the sense described in #40, with the difference that not all of the request is FORWARDed, but just a part of it, while the rest of the request is continued being executed by the current node (asynchronously with the forwarded sequence) ; a JOIN should resemble/reuse what we already implemented in the MULTI_FORWARD.

Practically, we might have:

COMPUTE 10ms
FORWARD_ASYNC next 3 commands (a)
  COMPUTE 5ms
  LOAD 256B
  REPLY 256B
FORWARD_ASYNC next 2 commands (b)
  COMPUTE 15ms
  REPLY 16B
JOIN a, b
REPLY 24B // final reply to the client

However, the JOIN in the above example might also need to be executed on a different node:

COMPUTE 10ms
FORWARD_ASYNC next 4 commands (a)
  COMPUTE 5ms
  LOAD 256B
  FORWARD_JOIN IP:port 256B
  REPLY 24B
FORWARD_F&F next 3 commands (b)
  COMPUTE 15ms
  FORWARD_JOIN IP:port 16B
  REPLY 24B

Here, the final "REPLY 24B" is the part to be executed on the joining node, IP:port, and it is replicated in both branches in the original request, however we might have a different syntax to denote this case. Moreover, the first FORWARD_ASYNC is meant to be non-blocking for the executing thread, that will continue executing the FORWARD_F&F in parallel; this latter command would be a Fire & Forget FORWARD, in the sense also discussed in #40.