search

Overv / outrun

Execute a local command using the processing power of another Linux machine.
Apache License 2.0
3.12k stars 64 forks source link

Re: README: suggestion for an elegant way to use SSHFS #14

Closed stapelberg closed 3 years ago

stapelberg commented 3 years ago

Thanks for this useful project!

In your README file, you write:

You may wonder why we’re not using an existing network file system solution like NFS or SSHFS. The trouble with these is that it’s difficult to automate quick setup for ad-hoc sessions. NFS needs to be set up with configuration files and SSHFS requires the remote machine to be able to SSH back to the local machine. In contrast, the included RPC file system is a lightweight TCP server with a per-session token that can be securely tunneled over the SSH session that we're already using. Having a custom solution also opens up opportunities for a lot of optimizations as we'll see later on.

I would like to make you aware of an elegant way to use SSHFS: please see the paragraph “reverse sshfs” in my article https://michael.stapelberg.ch/posts/2018-03-13-cpu/, where I describe that one can start sftp-server and connect it with an sshfs process running on the remote machine via SSH input/output redirection.

Perhaps you can switch to that and get rid of the custom FUSE file system, if it compares favorably performance-wise.

Overv commented 3 years ago

Nice project :) I searched for similar work for quite some time before getting started on this but I haven't come across your blog post before. Though, admittedly I didn't search for any Plan 9 related keywords.

The caching and prefetching that the FUSE file system implements make a huge difference for the performance of outrun, especially when using it over the internet. You can see this for yourself by specifying the --no-cache and --no-prefetch flags. I don't think the use of SSHFS would provide any advantages that would make up for this, especially since it's another dependency that would have to be installed.