cjbassi / gotop

A terminal based graphical activity monitor inspired by gtop and vtop
GNU Affero General Public License v3.0
7.39k stars 444 forks source link

FEATURE: Remote monitoring #132

Open ghost opened 5 years ago

ghost commented 5 years ago

I'm submitting this for discussion. I'd like to know if this would be considered in-scope for the project, and if so, what thoughts others have on how to go about it.

The use case is remote monitoring of multiple machines into a single, local, gotop instance display. If the user monitor 6 machines, it's crowded enough that User can't see all of the details in each on one display; User is left with either a) having them in tabs/screens, where they can see only one machine at a time, b) buying a much larger monitor, or c) having the output truncated. The solution would present input from multiple machines as virtual sensors on a single display; for instance, they would see CPU0-3 (local), 1-CPU0-1 (remote 1), 2-CPU0-1 (remote-2), and so on.

This would require three components:

  1. A communication protocol
  2. A mode in which gotop sends stats over the protocol
  3. A mode in which gotop receives (and displays) one or more streams

Naïve Implementation

The starting point would be choosing something like protobuf or MessagePack and modifying the (or adding?) widgets the to dump a stream instead of setting up and rendering on a display. For display, we'd add virtual widgets that take Readers and demarshal the input into virtual sensors for CPU, memory, etc., which could then be presented to the display widgets for display; Remote CPU0 would show up as CPU0-1 or something similarly brief.

Networking

Initially, for development, we ignore the client/server part; initial development is on Posix systems to take advantage of pipes and defer the networking for the moment (for reasons I'll get to below). Assume a wrapper start script for the local instance that sets up pipes and runs gotop with the pipe file handles as arguments; local gotop reads the input stream from these. The user is responsible for moshing into the remote server and running the headless gotop, piping the output to the pseudo-files.

Things to solve

CPU and memory are easy; everything is percentages, so there's no relative scaling. Other widgets will need decisions:

Security

Pipes are not a long-term solution; it doesn't work on Windows, and we don't want to break the fantastic "one binary" application. However, getting the network component correct will be challenging; it requires solving the authorization & transport encryption issues. There may be (is probably) a library that could be used to do this; I wouldn't want to have to roll this ourselves, but an ideal solution might hook into mosh or ssh to prevent adding yet more authentication and firewall set-up. It requires further investigation, anyway, but as it's a solvable problem I'd fork it off into a separate conversation.

cjbassi commented 5 years ago

Very interesting, and I appreciate the write up. I know ksysguard allows for remote monitoring by starting a remote program either at boot or when a connection is established, and it defaults to using ssh. However, it's a gui program and is more difficult to run on a remote host than gotop.

So it seems like the 2 main issues are:

I personally think using multiple tabs would be good idea, but at that point, I'm wondering if it would be 'good enough' to just use tmux, and set it up using a bash script that automatically connects to the hosts and opens gotop. If that turns out to be a complex script, then I would be fine with including that in the repo for people to use. I haven't tried this though so I'm not sure how convenient/inconvenient that would be. But it would also allow for a more robust method of customizing the layout/tabs.

There's also a question of how popular of a feature this would be, and whether implementing such a large feature would be worth it for enough people.

So I guess I'm leaning more so against implementing this, although it is a cool idea and it is something I've considered. I don't have much experience with monitoring multiple remote systems though, and I'm open to other thoughts on this.