Logging

[BenediktBurger]

How do we implement logging?

Using the Dara protocol

• do we have to subscribe again after any new Component joined?
• how to deal with the "slow joiner symptom" : does a log entry of a recently created Component arrive at the logging facility?

How to distinguish it from normal data?

• different ports? (my favorite)
• different topic?
• different payload (creates more traffic)

Using command protocol

• how does the Component know, to whom to send the log entry?

[bilderbuchi]

What about

All Components publish logs to a common connection at the Coordinator (irrespective of data or control protocol). Interested parties subscribe to that connection. Logs are structured by Component ID and log level.

Is this a viable approach? Where do we have needs in conflict with that?

[BenediktBurger]

All Components publish logs to a common connection at the Coordinator (irrespective of data or control protocol).

Control and Data Protocol use different socket types, therefore, they require two different addresses (host port combination).

The (Control Protocol) Coordinators could serve also as (Data Protocol) proxy and publish (data protocol) log entries on behalf of their Components which arrived via Control Protocol.

However, the advantage of pure Data Protocol Proxies is, that they are simple and virtually indestructible programs.

We could make the Controller publish (via Data Protocol) log messages arrived from the Control Protocol:

sequenceDiagram
    Note over ComponentDEALER,ComponentPUB: Different protocols of the same component
    ComponentDEALER ->> Coordinator: R:"C1.Coordinator". S: "C1.Component". Log "xy".
    Note over Coordinator,CoordinatorPUB: Transfer in Code
    Coordinator ->> CoordinatorPUB: Log "xy" on behalf of "C1.Component".
    CoordinatorPUB ->> LogProxy: S:"C1.Component". "xy".
    ComponentPUB ->> LogProxy: S:"C1.Component". "xy.

Then any Logging facility may connect to the LogProxy and subscribe the messages.

[bilderbuchi]

I think my understand of the PUB/SUB data protocol side is still spotty. I thought the Coordinators would handle that, too. So, a Coordinator would contain a "LogProxy", would subscribe to all its Components data connections and XPUB (is that the word?) that so that other interested Components could subscribe the log stream for a whole node at the Coordinator, and filter as desired. I did not think a separate entity would do that.

The same mechanism in parallel would work for logging, too. Maybe?

[bilderbuchi]

We also have to decide what "logging" is.

• Is this just for self-emitted messages from any components?
• What if a component emits/receives an ERROR messages on the control protocol? Is it then free/obligated to publish that as a log message, too?

I thought of logging, similar to how Python handles it, that Loggers blast messages into the void, for whomever to handle/read. And the Coordinator is a central point collecting all node-local log messages, for easy collection (to avoid NxN subscriptions between all Components).

[BenediktBurger]

Yes, we could combine the PUB-SUB proxy (the ports are XPUB, XSUB, as they hand on everything, including subscriptions) with the Control protocol Coordinator in one larger piece, which does both. But both parts are independent and do not share anything.