Add "rsync-ext" repository type which calls external rsync command

The existing rsync repository type uses execnet's emulation of rsync-like file synchronisation functionality in order to upload the repository to the remote system. However, execnet's implementation has some limitations, most notably that it does not synchronise deletions: files which are deleted in the local copy of the repository are not removed in the remote copy. This can cause surprising behaviour, for example when working with conditional inclusion in templated files.

This change introduces a new repository type, which I've called rsync-ext ("external rsync"), which invokes rsync as a subprocess in order to synchronise the local working copy of the repository with the remote system, instead of relying on execnet. rsync is invoked with the --delete flag, which means that files in the remote copy which do not correspond to the local copy are deleted, and with a list of exceptions so that local files related to e.g. version control and appenv are not synchronised.

This change also includes some changes to the handling of privilege escalation when connecting to a remote host as an intermediate step. batou automatically detects if the login user for a remote host is not the same as the deployment user, and invokes sudo as appropriate. Prior to this change, this was achieved using a shell snippet which gets executed on the remote system which prepends the sudo commands if a mismatch in login and deployment user is detected. As this is handled entirely on the remote system, this means that batou itself is unaware of whether privilege escalation was required.

I've replaced this shell snippet with logic within batou which calls the remote_core to detect login/deployment user mismatches and then reconnects to the remote host in case of mismatch. Whether or not sudo is required is recorded in a class member, so that other parts of batou can use this information, and so the autodetection can be skipped for future connections. There is also a corresponding require_sudo environment configuration option to explicitly specify whether privilege escalation is required instead of relying on autodetection and a possible reconnection.

rsync-ext needs to ensure that the spawned rsync process invokes the appropriate command on the remote host if sudo is required. With the extended privilege escalation detection code, it only needs to read an object attribute and add the --rsync-path command if required.

I haven't been able to get the test suite working properly on my local machine to check for regressions, however I've manually tested these changes with a dummy deployment (using both rsync-ext and the remote sudo detection code), and everything appears to work correctly.

flyingcircusio / batou

Add "rsync-ext" repository type which calls external rsync command #399