wahl-sec
commented
2 years ago In order to implement this we should implement a communication protocol between Daemons to use, preferably using something like gRPC to communicate using RPC calls.
On creation a daemon opens up for communication over a certain predefined port on the host allowing certain RPC calls, using these RPC calls Daemons can communicate with each other. The RPC calls should support creating a runner with a certain plugin or configuration on a another daemon and get the results for the task.
Example:
Involved Parties:
- Daemon A (Host A)
- Runner A.1
- Runner A.2
- Daemon B (Host B)
- Runner B.1
- Runner B.2
Goal: User wants to execute a plugin determining the largest files on the filesystem for each host. User runs Daemon A on the local system and executes the plugin on any of Daemon A's runners (Runner A.1, Runner A.2). However, the user does not want to copy over the plugin to the remote system running Daemon B and manually run the plugin there and then copy over the results back to Daemon A. So the goal for the user is to create a config that defines which Daemon's to run the plugin on by setting a value in the config, like daemons: [{"host": "[HOST]", "port": "[PORT]"}, ...].
Process:
When the plugin has been parsed, before we start our local runner for the plugin we go through the daemons list if present and check if the daemon is open and ready to receive a task, we send the task as the raw config of the plugin without the daemons section present as a JSON object using gRPC. If we have a custom made plugin in Python then we need to serialize the plugin using by pickling it or something similar. We should look into how the custom plugins are stored on the receiver daemon, perhaps load them inside the python namespace to have them only exist in memory and import them as such to avoid having to store the plugins on the receiver filesystem. When sending the plugin we also send a random UUID connecting the receiver to the sender daemon. The receiver daemon can then execute the plugin as a normal plugin. Any results collected will be sent back to the sender daemon using gRPC with the UUID and the sender daemon can append the data to a data store it keeps for the receiver daemon.
Security:
In order to secure the communication we need to use TLS certificates which gRPC has support for, storing the private key on the sender daemon and setting up secure channels for the two daemons to speak with.
Securing the pickling of the data in order to avoid serialization attacks we should sign the object using HMAC as suggested by the pickle module. The HMAC can actually be the UUID if we want to save some space. (Note: we should also sign the JSON configs so we know that they haven't been altered either so the HMAC will always be in the request even if we dont pickle any data)
Optionally:
We need to solve the issue of not knowing the host address and port of the daemons on other systems. One method is to have a central daemon that we know the IP of (or create a dns record to) then have the other daemons ping that central daemon with their name and configuration during certain timeframes, the central daemon can then keep track of the host and port for each daemon.
Easiest is to just create a dns record for each host daemon though.
Support functionality for having the daemon send out plugin as a task to child runners on other hosts ("or same host"). This could work by having the daemon send the plugin ("python file") together with the arguments encrypted to the child runner and then have the child runner execute and send back results encrypted to the daemon.
The encryption process should be made by having a children notify on creation ("or on request") to the daemon their public key, the daemon then responds with their public key and then they can individually create a secret key used for the communication process.