Documenting Teraslice messaging

[busma13]

This issue is just a place to document teraslice-messaging communications. Sequence diagrams are generated using mermaid-js.

Class inheritance

cluster-master/Client extends messenger/Client extends messenger/Core extends EventEmitter cluster-master/Server extends messenger/Server extends messenger/Core extends EventEmitter

execution-controller/Client extends messenger/Client extends messenger/Core extends EventEmitter execution-controller/Server extends messenger/Server extends messenger/Core extends EventEmitter

List of messages for each relevant function within all messaging classes

send, sendToAll and emit will emit messages handle response, once, onceWithTimeout and on will listen for messages

cluster-master/client.ts: { 
    send: {
        execution:finished, cluster:analytics
    }
    handleResponse: {
        execution:pause, execution:resume, execution:analytics
    }
}
cluster-master/server.ts: {
    send: {
        execution:pause, execution:resume, execution:analytics
    }
    handleResponse: {
        execution:finished, cluster:analytics
    }
    emit: {
        execution:finished, cluster:analytics
    }
    on: {
        connection, execution:finished
    }
}
execution-controller/client.ts: {
    handleResponse: {
        execution:slice:new, execution:finished
    }
    emit: {
        execution:slice:new, execution:finished 
    }
    send: {
        worker:slice:complete
    }
    once: {
        execution:slice:new
    }
}
execution-controller/server.ts: {
    on: {
        connection, slice:success, slice:failure
    }
    send: {
        execution:slice:new
    }
    sendToAll: {
        execution:finished
    }
    handleResponse: {
        worker:slice:complete
    }
    emit: {
        slice:failure, slice:success, worker:enqueue
    }
}
messenger/client.ts: {
    on: {
        error, server:shutdown, reconnecting, reconnect, disconnect, shutdown, message:response, connect_error
    }
    once: {
        connect_timeout, connect, reconnect
    }
    emit: {
        ready, <msg.id>
    }
    send: {
        client:available, client:unavailable, client:shutdown
    }
}
messenger/server.ts: {
    on: {
        connection, client:online, client:available, client:unavailable, client:disconnected, client:shutdown, client: reconnect, client:error, error, disconnect, message:response
    }
    emit: {
        ready, shutdown, client:reconnect, client:<state>, client:online, client:error, client:shutdown, client:disconnected, 
        client:available, client:unavailable, <msg.id>, connection
    }
    handleResponse: {
        client:available, client:unavailable, client:shutdown
    }
}
messenger/core.ts: {
    on: {
        error, <eventName>
    }
    emit: {
        message:response, <eventName>, <eventName:msg.scope>
    }
    onceWithTimeout: {
        <clientId>, ready:<clientId>, ready<serverName>
    }
}

TODO: add styling to diagrams to make things clearer and more legible.

[busma13]

    sequenceDiagram
        box transparent Pause/Resume Execution
        participant  Master
        participant Execution Controller
        end
        rect rgb(30,30,30)
        autonumber
        note over Master, Execution Controller: Pause
        Master->> Master: setup onceWithTimeout(msg.id)
        Master-->>+Execution Controller: emit('execution:pause', msg) 
        Execution Controller->>Execution Controller: on('execution:pause', msg)<br/>tell scheduler to pause<br/>emit('message:response', msg)
        Execution Controller-->>- Master: emit(msg.id)
        end

        rect rgb(30,30,30)
        note over Master, Execution Controller: Resume
        Master->> Master: setup onceWithTimeout(msg.id)
        Master-->>+Execution Controller: emit('execution:resume', msg) 
        Execution Controller->>Execution Controller: on('execution:resume', msg)<br/>tell scheduler to resume<br/>emit('message:response', msg)
        Execution Controller-->>- Master: emit(msg.id)
        end

[busma13]

        sequenceDiagram
        box transparent Cluster Analytics
        participant  Master
        participant Execution Controller
        end
        rect rgb(30,30,30)
        autonumber
        note over Master, Execution Controller: EXC.run()
        loop setInterval()
        Execution Controller->>Execution Controller: create analytics diffs obj
        Execution Controller->>Execution Controller: setup onceWithTimeout(msg.id)
        Execution Controller-->>+Master: emit('cluster:analytics', msg)
        Master->>Master: on('cluster:analytics', msg)<br/>update clusterAnalytics.controllers<br/>emit('cluster:analytics', { diff, current})<br/>emit('message:response')
        Master-->>-Execution Controller: emit(msg.id)
        Execution Controller->>Execution Controller: Update pushedAnalytics object
        end
        end

        rect rgb(30,30,30)
        note over Master, Execution Controller: EXC.shutdown()
        Execution Controller->>Execution Controller: clearInterval()
        Execution Controller->>Execution Controller: create analytics diffs obj
        Execution Controller->>Execution Controller: setup onceWithTimeout(msg.id)
        Execution Controller-->>+Master: emit('cluster:analytics', msg)
        Master->>Master: on('cluster:analytics', msg)<br/>update clusterAnalytics.controllers<br/>emit('cluster:analytics', { diff, current})<br/>emit('message:response')
        Master-->>-Execution Controller: emit(msg.id)
        Execution Controller->Execution Controller: Finish Shutdown
        end

[busma13]

        sequenceDiagram
        box transparent Execution Analytics
        participant API
        participant Master
        participant Execution Controller
        end
        API->>+Master: GET<br/>'/txt/slicers'<br/>'/txt/controllers'<br/>'/cluster/slicers'<br/>'/cluster/controllers'<br/>'/jobs/:jobId/slicer'<br/>'/jobs/:jobId/controller'<br/>'/ex/:exId/slicer'<br/>'/ex/:exId/controller'
        Master->>Master: setup onceWithTimeout(msg.id)
        Master-->>+Execution Controller: emit('execution:analytics', msg) 
        Execution Controller->>Execution Controller: on('execution:analytics', msg)<br/>add analytics to msg.payload<br/> emit('message:response', msg)
        Execution Controller-->> Master: emit(msg.id)
        Master->>-API: response