Closed DavidPL1 closed 5 years ago
Scratch all that. The python logging module supports SocketHandler to send logs over socket and MemoryHandler to buffer logs until an output of a certain level is emitted or the buffer is full (the buffered log is then flushed to another handler).
New approach: Collect all logs on the server by running SocketHandlers in combination with MemoryHandlers on remote hosts. On the UI machine use sshfs to create a symlink of the servers log location to access remote files with local file descriptors (super network efficient: sshfs with tail on a changing file only causes traffic when the file is changed)
When a UI is not started in standalone mode and not running on the same machine as the server, log files need to be synced in order to be displayed. Copying a file is less of a problem than how: the philosophy of this application is optimizing resource usage, in this case keeping network traffic at a minimum, that's what this issue is about.
Possible optimizations:
Proposed approach:
This could also be applied to slave managers in #24 and the server could even forward log requests to the slave managers to directly connect the remote machine running the slave instance to machines running a UI instead of gathering remote logs and forwarding them to subscribers (major reduction of traffic on the server)
Maybe also interesting for #18