Streamline server install from client connection

[ghost]

Scenario: User has the mosh client installed and wants to connect with it to a server that doesn't have mosh-server installed. Furthermore, user doesn't have admin privileges on server.

Desired: running 'mosh server-location" looks for the mosh-server binary. If none found, it prompts the user to install mosh on the server locally. Then the mosh-server binary for the current architecture is downloaded and stored locally (say "~/.mosh/mosh-server" or "~/.local/share/mosh/mosh-server") and mosh-server is started.

Current: To install a workaround is needed which is a huge pain.

Workaround (all plaforms): Connect through ssh. Install mosh locally on the server: Get all dependencies (ain't happening.. not without admin): Protocol Buffers protobuf-compiler, libprotobuf-dev Boost libboost-dev utempter libutempter-dev ncurses libncurses5-dev zlib zlib1g-dev IO::Pty Perl module libio-pty-perl $ wget https://github.com/downloads/keithw/mosh/mosh-1.1.3.tar.gz $ tar -zxvf mosh-1.1.3.tar.gz $ cd mosh-1.1.3 $ ./configure $ make (mosh-server is in .src/frontend/make-server) exit mosh --sever=/path/to/mosh-server server-location

Workaround Ubuntu 11.10 (or for other distributions go to https://launchpad.net/~keithw/+archive/mosh/+packages and wget the correct .deb distribution package): Connect through ssh. $ wget https://launchpad.net/~keithw/+archive/mosh/+files/mosh_1.1.3-0~585~oneiric1_amd64.deb $ ar -vx /mosh_1.1.3-0~585~oneiric1_amd64.deb $ tar -zxvf data.tar.gz $ cp /usr/bin/mosh-server . $ exit mosh --sever=/path/to/mosh-server server-location

[ghost]

Useful discussion from #mosh:

kmc: oh, another problem with ~/.mosh/mosh-server: some users have a network-shared filesystem across multiple OSes / architectures that's why xmonad compiles itself as ~/.xmonad/xmonad-x86_64-linux so we'd need to add logic for that

[xmw]

s/architecture/(ditribution, architecture)/ as fixed-release distros use different library ABI versions.

You'll need all runtime versions of the dependencies (shared object) like libutempter, libprotobuf, too.

scanelf -n /usr/bin/mosh-server

TYPE NEEDED FILE ET_DYN libncurses.so.5,libprotobuf.so.7,libz.so.1,libpthread.so.0,libutil.so.1,librt.so.1,libutempter.so.0,libstdc++.so.6,libm.so.6,libgcc_s.so.1,libc.so.6 /usr/bin/mosh-server

[directionless]

I'm against having mosh upload a binary. Pragmatically, I think it will be annoying to maintain a slew of binaries for all platforms and I think it will be hard to chose the right one.

I think there's also question about how painful this upload would be on some lossy network connection.

Needing to have pre-installed mosh-server fits my usual pattern for this class of software.

[xmw]

I totally agree on that on a general basis It would be a nice feature to have the option on underprivileged accounts w/ non-responding admins (like computer pools).

[flying-sheep]

Me too. The common case right now is a common server (Debian, centos, Ubuntu) with an older version than the newest and mosh not installed (neither some dependencies)

So it would be awesome when a fully statically linked version of the server would be available for upload via simple mosh-parameter or automatically (with maybe-you-want-to confirmation dialog)

The argument about it being a pain to upload is a strawman: you have to upload or install it anyway and this bug is just about another, more convenient option to upload it: try to upload it manually over a lossy connection, that will be even more of a pain, yet you'll have to do it as underprivileged.

[notorand-it]

I think this is not really doable. And shouldn't be implemented.

First, I think that installing any software automatically at the connection is going to be considered as an attempt to circumvent the system administration policies. Unless explicitly allowed. The proper way is you to ask the sysadmins to install mosh.

Second, the "static binary payloads" should already exist on the connecting client. The number of different binaries needed on the client would be rather large depending on architectures and OSs, at least. Bash/Perl/Python/whatever_else "non binary payloads" would rely on dependencies to be pushed from the client as well. Again, run time version mismatch could create incompatibilities and would require multiple versions to be available on the clients. You cannot rely on the fact that the target has full internet access to download the missing bits. And you cannot rely on the fact that the client has it.

Third, if you already have SSH credentials and the firewalls aware of the mosh networking needs, then you'd very likely should be able to get mosh installed on the targets. If such networking needs are met "by chance" or "inadvertently", then your plan looks very close to illegal system access.

Fourth, I think that something that would make some sens is an operation mode to fall back to older version protocols, provided that this makes any technical sense.

Finally, in my humble opinion, this request makes no sense at all. Unless there's some strong demonstration in an opposite direction. Something like recurring uses cases I am missing at the moment surely due to my faults.

[haarp]

First, I think that installing any software automatically at the connection is going to be considered as an attempt to circumvent the system administration policies. Unless explicitly allowed. The proper way is you to ask the sysadmins to install mosh.

That's why it shouldn't be installed, but merely transmitted and executed. No part of it needs to reside on permanent storage. Just like any code or script you'd send with ssh.

Second, the "static binary payloads" should already exist on the connecting client. The number of different binaries needed on the client would be rather large depending on architectures and OSs, at least. Bash/Perl/Python/whatever_else "non binary payloads" would rely on dependencies to be pushed from the client as well.

Not necessarily. Not if it was written in plain Python. There's no requirement to use dozens of libs.

Third, if you already have SSH credentials and the firewalls aware of the mosh networking needs, then you'd very likely should be able to get mosh installed on the targets.

In some cases I may have the rights, but not the will to install additional software on numerous servers, adding yet another piece that needs to be maintained, kept updated, and synchronized between machines.

If such networking needs are met "by chance" or "inadvertently", then your plan looks very close to illegal system access.

That sounds very far-fetched. I'd even go so far to say that most personal Linux servers are already exposed to the internet with no firewall but the kernel's. Thus all that's needed to meet the networking needs is run a process on the corresponding port.

Additionally, in such an illegal case, mosh would not provide any more access than ssh already does.

Finally, in my humble opinion, this request makes no sense at all. Unless there's some strong demonstration in an opposite direction. Something like recurring uses cases I am missing at the moment surely due to my faults.

I may have permissions to run mosh, but not the permissions to install it. I may have permissions to install mosh, but not the will to do so and maintain it. The target may not have mosh in its package repositories and I may not want to gunk it up by installing things untracked by its package manager.

My goal would be to be compatible with any ssh-capable server, regardless of other software installed on it or installable on it. A just-works-anywhere solution.

[notorand-it]

That's why it shouldn't be installed, but merely transmitted and executed. No part of it needs to reside on permanent storage. Just like any code or script you'd send with ssh.

One thing is sending a few lines of shell script for an automated task over ssh, one thing is pushing the entire code for a mosh server (or whatever else). Isn't it? While the former could sneak through the system administration policies, the latter will be surely seen as a non compliant action. As far as my 30+ years experience as both user and system administrator.

Not necessarily. Not if it was written in plain Python. There's no requirement to use dozens of libs.

Just python (as well as any other interpreter): that will be the dependency (cascaded). On Debian stable mosh relies upon:

• libprotobuf
• libssl
• libtinfo
• libutempter
• zlib

plus a bunch of "normal" C/C++ dependencies from GCC. I presume the python implementation you are thinking of would either re-implement everything "in line" within the payload, or rely on a similar form of dependencies to be installed on the target.

In some cases I may have the rights, but not the will to install additional software on numerous servers, adding yet another piece that needs to be maintained, kept updated, and synchronized between machines.

That's the purpose of "system administration": to keep systems (hundreds, thousands of them) up and running and up to date. Again, if you are the sysadmin, very likely you are not manually updating them. If you are not, I hardly believe that bypassing the sysadmin rules is something that will be seen as "nice". And you still need the firewalls to be changed to allow for a range of (high) UDP ports to be opened: I don't think that firewalls grant server access over ranges of UDP ports as a default.

That sounds very far-fetched. I'd even go so far to say that most personal Linux servers are already exposed to the internet with no firewall but the kernel's. Thus all that's needed to meet the networking needs is run a process on the corresponding port. Additionally, in such an illegal case, mosh would not provide any more access than ssh already does.

If those servers are among your own belongings, then you surely already know about proper (manual, semi-automated and automated) system administration. I am excluding here any toy system, though. SSH over TCP:22 is one thing, MOSH over UDP:60000-60999 is another one. Isn't it? SSH server can be controlled and fine tuned by sysadmins, a MOSH server sent via SSH won't.

I still think that real life use cases for an ssh-sent python mosh server are very limited in number and usefulness. Good luck with you project, anyway.

[cgull]

For the record: I can't speak for all the project developers, but I'm not interested in trying to add something like this to Mosh. It is impossible for a client to fully know exactly what the server is, having the client bundle server binaries for other OSes becomes a nasty MxN distribution problem, and it just sounds like a support nightmare all around.

I would consider adding a hook to the Mosh script to copy a file over the initial SSH connection, if someone comes up with a clean patch for that. Then users can easily write scripts to implement this functionality locally for their specific situation. It occurs to me that the combination of the ControlMaster option and scp/sftp might be useful here.

[eminence]

If someone wanted to try to build a fully static mosh-server binary (perhaps using musl), that would be a significant advancement. Even if it only worked on linux, I suspect it would be valuable. This is non-trivial to do, so having some type of automated build script to do this would be highly valuable.

[notorand-it]

I think @haarp is talking about non-binary payload. FWIM, there's no difference in dependencies between binary and non-binary, with the former situation possibly much simpler with statically linked binaries.

I am still dubious with the idea of introducing a hook to send something to be executed in the target: mosh client can be asked to execute remote commands just like ssh does:

mosh [options] [--] [user@]host [command...]

[haarp]

A binary, even if statically linked, still requires several versions for each platform and architecture. Not to mention that it would be huge. A python script, in comparison, would be relatively small, and it's reasonable to assume that every environment will already have a Python interpreter and its dependencies.

[javabrett]

Could we decompose this to:

1. Having a mosh-server server executable available, which can not only run unprivileged (as it does now), but can run regardless of target-system dependencies (other than an SSH server). This is currently not possible because the build for all platforms is a dynamically-linked binary, and it will often be that the dependent libraries are not installed on the system, and that typically requires root access.
2. Selecting and transporting such a portable mosh-server suited to the target platform, onto the target server at least once, in a location executable by the unprivileged user.

1) requires either a statically-linked binary suitable for the target system, or a script/non-binary server, or some magic to allow the unprivileged user to resolve the libraries (unlikely without some weird chroot). 2) seems a lot less important and separating it removes a lot of complexity, since we no longer have to worry about automatically selecting the correct mosh-server for the target platform - this becomes the responsibility of the user - and choices such as fetch/download or upload-from-client are no longer needed. Since we have ssh access then an upload option is available, and ssh-and-curl options should also be an option for internet-connected hosts.

So if there was a recipe for a couple of statically-linked binaries for common platforms, regardless of whether the mosh project chose to build and distribute those, there would be a no-remote-install option for certain platforms. The significant downsides of statically-linking binaries acknowledged and aside. Also assuming that such system access isn't in violation of expected use of the host, and noting that UDP port access is still required.

I took a stab at a statically-linked binary mosh-server. I chose Alpine Linux as the build platform, because a) it uses musl natively as its libc, and it is static-friendly, b) it's tiny and c) there's a Docker image. My Dockerfile installs a compile sdk, all the normal mosh build-deps, plus ncurses-static (static libs are absent in its main package), and an unneeded but useful for testing openssh server. I then built mosh with LDFLAGS="-static", created a mosh test-user and started sshd. Can then use a normal mosh client to connect to this, as-expected, the only difference being the server is statically-linked and with an alternate libc.

docker build -t alpine_mosh_static dockerfiles
docker run -itd --name alpine_mosh_static1 alpine_mosh_static
docker inspect alpine_mosh_static1 | grep IPAddress
mosh mosh@172.17.0.2

I copy mosh-server from this image to local. Unsurprisingly it is 12Mb instead of the normal 300-400k.

docker cp alpine_mosh_static1:/usr/bin/mosh-server /tmp

I then created an image based on ubuntu, Dockerfile.ubuntu which installs openssh-server, adds a mosh test-user and starts sshd. I can then scp the statically-linked mosh-server to the mosh user's home directory (step 2 above), then I can mosh-in to the server, without having installed any additional libraries:

docker build -t ubuntu_mosh_static_test -f Dockerfile.ubuntu dockerfiles
docker run -itd --name ubuntu_mosh_static_test1 ubuntu_mosh_static_test
docker inspect ubuntu_mosh_static_test1 | grep IPAddress
scp /tmp/mosh-server mosh@172.17.0.3:~
mosh --server="~/mosh-server" mosh@172.17.0.3

... and can mosh-in without having mosh-server centrally-installed, and without libprotobuf or other usual dynamic-link dependencies being installed on the target.

Does this help at all, in terms of a recipe that could be published for building static-binaries, or indeed incorporated into builds? Next steps could be to stress-test the static mosh-server, or test additional common platforms to see where it will run, where it won't, and what other builds might be helpful.

[notorand-it]

Nice. You only forgot, in my opinion, a few prerequisites:

0. Scp is allowed on the target.

1. The client knows where a writable directory is. Home could not be one.

2. Execution permissions will be allowed on payloads.

3. The firewalls and the routers between client and server need to allow for that UDP traffic in either direction.

4. Server admin policies allow for payloads to be transfered and executed.

Of course, if you are "working" in a simple local (or virtual) environment, chances there are that those requirements are met. For real world scenarios it's a completely different story, especially for points no.3 and 4.

[crides]

I'm kind of surprised that this thread has been silent for this long...

3. The firewalls and the routers between client and server need to allow for that UDP traffic in either direction.

This is actually the biggest problem when I was testing mosh on my university's server. As we can scp files to and access the Web from the server, building the dependencies are actually not that hard, although many other situations may not have the same ease of access. However, we can at least simplify the process of building mosh itself by building AppImage's for different architectures.

[dtinth]

I created an automated x86-64 static binary distribution. Details is in this issue:

• https://github.com/mobile-shell/mosh/issues/1006#issuecomment-909227128