Running tool on remote node, tgz'ing exported files & transferring them to admin node

[jecluis]

As discussed with @jan--f on irc, this is mostly meant so that we don't forget we had this discussion.

The whole idea would be to be able to run a given tool on all remote OSD nodes. Said tool would grab given data out of the OSD (e.g., maps), and export them to a predefined (or user-defined) directory; directory would be compressed, and then transferred to admin node.

This could potentially be useful for monitor rebuild, during a disaster recovery event.

[jan--f]

I can see this working with https://docs.saltstack.com/en/latest/ref/modules/all/salt.modules.cp.html#salt.modules.cp.push

The cp module is deactivated by default though...'for security purposes'. Might be ok to activate it though.

[swiftgist]

Is this a preventative strategy? As in, the files would not be pushed to the monitor unless something bad happened?

We would need to experiment but there may not be much need to write the maps to a file. A salt module could return the json output of the maps. If needed, the module can be added to the salt mine to cache the data. (This can be forced refreshed as needed.) A salt runner could collect the data from all the minions and then optionally push the data to the monitor. If we have the equivalent shell script steps, this wouldn't be too difficult.

The beneficial part of this separation is that each minion can be inspected independently (e.g. salt '*' cephosd.maps or similar) before attempting to do anything with the data.

[jschmid1]

We have supportutils-plugin-ses and supportutils-plugin-salt which essentially reaches out to the nodes and scrapes data, should we maybe just include mon-maps etc in those scripts?