Tether restructure

[caglar10ur]

As a developer reading the source code of the project, I need to understand how tether works, its internal details, plugin architecture and interaction component. I also would love if reading the tether source code could be little bit easy on eyes.

Work items

• [x] Iterate tether functional & non-functional requirements
• [x] Figure out primary issues with the current code
• [ ] Figure out the issues with the current design
• [ ] Specify acceptance criteria for tether design update (George)
• [ ] Refresh tether design docs (George)
• [ ] Document the future plans for tether
• [ ] Consider different approaches (runs as init, runs under systemd etc.)
• [ ] Document the new design

[hickeng]

I've added high level functional and non-functional requirements here: https://github.com/vmware/vic/wiki/Tether

[vburenin]

Does current tether implementation support connect/reconnect?

[corrieb]

Note to add here to highlight the importance of systemd integration. It's something that was highlighted by the SABU team doing the Kubernetes integration, that it forced them to use an older version of Debian that uses init.d. It's something that will continue to come up the more we expect containerVMs to run more multi-process / multi-app workloads.

Problem is more about the expectation of applications and scripts running on a particular version of Linux that they detect whether or not they should use systemd simply by looking at the version, rather than checking whether it's actually running. This is not an unreasonable assumption.

Speaking to @mlh78750 about this, he seems to feel that we can probably run systemd in a process namespace as PID 1 and this may be a short term fix, rather than having to integrate the tether tightly with systemd.

[caglar10ur]

@vburenin it supports reconnect, yes

[mlh78750]

@corrieb pointed me here when we were talking about systemd earlier. We should be able to use the process namespace for the runtime/entrypoint to allow init/systemd style inits to work even when the tether is actually pid1. Just start another process namespace and exec the entrypoint in that namespace. This should allow everything in the container to look like it does when it's a container in vanilla docker.

[caglar10ur]

@corrieb IMHO the expected way to run multi process/multi app workloads is using tether and configuring multiple sessions and not using systemd.

[mlh78750]

@caglar10ur if we're using the port layer as a general IaaS layer and we want to run VM's that are packaged in docker image format (ie. not containers) then allowing those VM's the ability to have their own init process is very important.

[vburenin]

If we are limited by serial port performance in the middle term (1 year or so), we should add gzip/bz2 compression for serial port channels.

[corrieb]

@caglar10ur There are some things which simply won't install or run without systemd. It can be as simple a matter of scripts expecting to call systemd or as complex as something like Rkt wanting to use systemd-nspawn. If we're expecting to run anything in a containerVM, I think we need it.

[hickeng]

@mlh78750 if we're talking about being able to provision VMs authored from images, without managing them via a VCH then we would also need to consider the standalone configuration requirements such as authentication, remote access, etc. In this case we are using a VCH solely to provision and if that's really needed then we could have an OS variant that runs systemd instead of tether as the init process.

[mlh78750]

I should be more careful and more pedantic with this.

Yes, there will be work in the future to do more of an IaaS API/personality for the port layer.

No, I'm not saying tether needs to be the design as to how we accomplish this.

originally I was just simply trying to add that using process namespace would make the entrypoint process pid 1 and would make the container as a VM look like a container. It's 2 extra system calls when launching the entrypoint and as a byproduct, the k8 poc that the storage team did would have been easier if the entrypoint for the docker images they created were able to run as pid 1 (which is the systemd issue @corrieb pointed out)

I think that the customer value of adding a namespace for entry point is high regardless of issues with systemd or other init systems.

[caglar10ur]

I've a branch with some documentation (https://github.com/caglar10ur/vic/tree/tether-ng) to start talking about new re-organization.

[caglar10ur]

To clarify, the branch has some additional files but they are virtually worthless :) I just copied the current tether interfaces and generated mocks to see how we could leverage gomock.

https://github.com/caglar10ur/vic/commit/4bf02c218a0f781417688a20f51749de71983b7a#diff-35d6291aaa728d895841ecf93170341f is the markdown file which talks about it. Again nothing is new about it, it's just trying to re-organize things with no real change. Please see this as an appetizer not a main dish :)

[casualjim]

• tether should not mess with the network
• tether should be able to provide attach capability
• tether should be able to provide IP address reporting
• tether should be able to provide secure exec functionality without the network
• allow for logs via syslog and/or serial port
• allow for templates to be applied based on values in guestinfo etc

In a world where tether would run as non pid 1 it might make sense to provide separate services that provide the necessary functionality.