This was a major refactor to make the comms system faster :rocket:
Major Changes:
Renamed worker to agent and minion to worker.
We now use a binary protocol for all messages.
Extracted cpp worker to its own separate repo.
Requesters cannot specify job types. Batch jobs uses all job types.
Single address for messages. Before, we used a vector<string> for messages, but we never actually needed more than one address.
Heartbeats
Heartbeats are negotiated with a HELLO/WELCOME exchange. Before, there were 3 magic numbers for heartbeat settings per router. Now it's only 1 number: how long before a timeout. Poll rates are automatically determined from the heartbeat timeouts.
Heartbeats run in the same thread as the rest. Less context switches and no need for running flag.
Heartbeats run at the socket level, so it's somewhat transparent.
Routers now use CRTP to statically dispatch messages to message handlers. No more router.binds.
Added lots of tests. The interfaces are also more testable now.
Stuff left to do:
[x] Job types for agent.
[x] Add tests for heartbeating
[x] Add tests for timeouts
[ ] Add tests for sad paths for delegator and agent
[x] Simpler job scheduling
[x] Add server bin
[x] Remove ununsed files
[x] Remove zmq_addon since we don't use it that much.
[x] Use meta::apply for router poll to avoid function lookup.
[x] Syntactic sugar for serialising and deserialising.
This was a major refactor to make the comms system faster :rocket:
Major Changes:
vector<string>for messages, but we never actually needed more than one address.runningflag.router.binds.Stuff left to do:
meta::applyfor router poll to avoid function lookup.