Programs like recvtty could use some descriptive data

[vanrein]

I understand that RecvTTY is just a demo, but RunC is out for a higher calling. And the detached style of using it is advised for security reasons over just using a foreground TTY. I was thinking of a more interesting RecvTTY and got stuck due to lack of information.

Would be awfully useful if a session management environment got more information than just the file descriptor for the TTY. This is possible and perfectly POSIX compliant.

Minimal information that could be useful might be the name of the container, along with the rootdir in which its state is stored. That way, such a session management environment might interact with the container, instead of just showing its TTY. Even if it'd just be able to show the name, but it might also add buttons for pause, start/resume and so on.

Trying to imagine how a user interface could control containers, this came up and made me realise that there is always a need for a second channel, which is silly because the primary channel can include it all in one atomic message. Can you please extend it like that?

[cyphar]

We used to pass more information in the pipe (the idea being that you could have one socket for all runc instances), but that was removed some time ago -- @crosbymichael do you happen to remember when and why this was done?

[vanrein]

We used to pass more information in the pipe (the idea being that you could have one socket for all runc instances), but that was removed some time ago -- @crosbymichael do you remember when and why this was done?

I don't remember, but noted in the RecvTTY code that it made no use of this facilty. What I have in mind is precisely to support this multiple-instance idea, and that is why I either need the extra data or setup new sockets all the time (and coordinate around it, do access control over and over again, and so on).

I want to incorporate support for this useful idiom in a session manager. I'm looking at TWIN, but the idea could carry over to Screen and Tmux too, and of course to desktops that choose to popup terminals whenever a new container is created. I think this can make container management a lot easier.

https://sourceforge.net/projects/twin/ https://github.com/cosmos72/twin

Specifically, I am hoping to find a way of integrating the RunC control (start/pause/resume/stop) as buttons in these terminals. I suspect that such a setup might also benefit from management simplifications like Rkt when they are built on top of RunC.

[vanrein]

I wrote down how I think we can have better TTYs for containers based on ASCII-encoded multiplexing:

• Multiple programs multiplex their stdio over one TTY
• This might include daemons that are started by init
• Split program streams for stdin, stdout, stderr
• Adding program streams for stdctl (start, pause, ...)
• Optional program stream for sasl to automate TTY login
• Optional program stream for zmodem for file up/download
• Optional program stream for syslog subject to selection to avoid logging inside containers

These should help to give control over a container, and at the same time simplify their internals.

Basically, I'm taking on the challenge that you are posing with your recvtty to really work it out much further :-) and I'm interested to learn if you agree this would be useful to improve container control.

[vanrein]

Oh, and...

• Adding program streams for stdctl (start, pause, ...)

...that's the one that would need container identity / root information to be able to exercise control to its outer level.

The rest is pretty much a way of communicating between container internals and externals. The only relation with RunC would be that this works better over SCTP socketpairs than the single-stream versions. It all works, but with less exploitation of the concurrency between streams. Think head-of-line blocking.

[cyphar]

I'm not entirely sure how many of these things would be required to be done within runc. As above, if there is a usecase for having information associated with the console sent through --console-socket I'd be more than happy to re-add the metadata. Here are a few short comments:

Split program streams for stdin, stdout, stderr

There is a strong argument for doing this with real consoles within runc (and I considered including it in "the great console rewrite of 2016", though it would require getting all of the people using runc to handle having two or three PTY streams) -- though it should be noted that not even terminal emulators or shells do this.

Adding program streams for stdctl (start, pause, ...)

I don't really know what stdctl is? Do you have a reference for this somewhere? Or is this a term specific to your project idea?

Multiple programs multiplex their stdio over one TTY

While I do think this (and the associated ASCII-encoded multiplexing setup as well as the program streams designs) is quite interesting, this isn't something that we would want to do inside runc. To be frank, just reading it is making me nervous about the security implications of re-encoding malicious code's output for the purposes of container management. But it would be interesting to see a project that builds on top of runc implement those kinds of features.

[vanrein]

Hi Aleksa,

Adding program streams for stdctl (start, pause, ...)

I don't really know what stdctl is? Do you have a reference for this somewhere? Or is this a term specific to your project idea?

This is something I am considering to introduce. A meta/control interface to do the things we currently use through signals, UNIX sockets, container management but that often does similar things in confusingly different manners. It was here for reference.

Multiple programs multiplex their stdio over one TTY

While I do think this (and the associated ASCII-encoded multiplexing setup as well as the program streams designs) is quite interesting, this isn't something that we would want to do inside runc.

Definately not! It is something the container would do internally, and then the outside. I mentioned it to clarify that it is useful to have some descriptive information, and not just a TTY file handle, to be able to talk to runc with the right instance information. The TTY handle does not give that instance information.

To be frank, just reading it is making me nervous about the security implications of re-encoding malicious code's output for the purposes of container management.

Well aware of any such issues; I am a cryptographer. The current practice of mixing login along with application data is what gets me jumpy and agitated.

The solution, of course, is to have this multiplexing layer orthogonal to applications, escaping on input and unescaping on output. That way, applications could not overtake any of the multiplexing streams -- accidentally or purposefully.

It's all pretty old-fashioned ASCII stuff, all I'm considering adding is some more structure that I think is worthy.

But it would be interesting to see a project that builds on top of runc implement those kinds of features.

Thanks, that's lovely to hear.

So, do I stand a chance with this request of some more data to travel along with the tty file handle? I think the root-dir and image name should suffice. I leave it up to you wheather you want to split stdin/stdout/sterr.

-Rick

[vanrein]

I don't know what the inner workings of RunC and its underlying containers are in this respect, but now that we're discussing what to deliver, would it be an option to use SOCK_SEQPACKET for the TTY socket?

To avoid channel overtaking, I'm distinguishing mulTTY connections, like the thick arrows in this image. To enter the thick bandwagon, escaping is to be used and to leave it unescaping is to be used. [It would also be safe to escape, escape, unescape, unescape if layering of software led to that.]

Atomicity of data is a basic requirement; otherwise one program might give rise to injecting data into another program's stream. I could get this working with write() and/or writev() by sending no more than PIPE_BUF bytes for mulTTY sequences, and/or using a mutex wrapper around it, but there I have a remaining concern about shared streams doing things like 2>&1 to introduce multiple data sources. With sendmsg() this is no concern, and especially on a channel designed to keep message boundaries in tact the guarantees are even stronger. That is what SOCK_SEQPACKET can do. It is available for UNIX domain sockets, socketpair() and plain socket() where it becomes SCTP.

Links:

• The RATIONALE on atomicity of write() for lengths up to PIPE_BUF on pipes.
• The specification for sendmsg is: If space is not available at the sending socket to hold the message to be transmitted and the socket file descriptor does not have O_NONBLOCK set, the sendmsg() function shall block until space is available. If space is not available at the sending socket to hold the message to be transmitted and the socket file descriptor does have O_NONBLOCK set, the sendmsg() function shall fail.