Weirdness in TTY

[jwendell]

1. Create a terminal-less container (runc create ...)
2. In another terminal, start it (runc start ...)
3. Go back to terminal 1 and press ENTER some times
4. See the weirdness

Notes:

1. I'm running the example from README (busybox image, "sh" entry point)
2. After start the container (step 2 above) I see in the first terminal: sh: can't access tty; job control turned off

[cyphar]

@jwendell If you've disabled terminal creation ("terminal": false in your configuration) this is to be expected. What's happened is that your container has inherited its IO file descriptors from your shell, and your shell is still using them. So when you enter anything into your shell there is a race between your shell and the container process that determines which process reads the bytes.

There are several ways to use runc and runc accomodates all of them, but if you use the wrong combination it will cause you problems. To be fair this is not a well-documented area because of how much stuff has changed in the past few months and we're still not sure if the current system is the best way of doing things.

terminal=false is an unfortunate option, but is generally used in cases where the parent process of the container has already set up the IO of the container and doesn't want runc to touch it. runc create/runc start (or runc run -d which is sort of like a combined version of those commands) results in the runc process no longer being alive afterwards. And due to the semantics of double-forking your shell will interpret that as "the child has finished running, it's my job to continue running now". If you're using just a raw runc create from a shell you're most likely doing something wrong. You should either use it with --console-socket, or set the stdio of runc create to files that you want to be linked to the container's stdio.

terminal=true is the nicer option to use, because it tells runc to create a new terminal inside the container's devpts setup. This results in much more sane behaviour from a variety of tools and it better emulates how a shell on a normal system acts. Unfortunately most users hit problems with it when they (mistakenly) try to use runc create in a shell with terminal=true. This won't work (unless you provide --console-socket which then requires you to use a separate program to capture the master pty file descriptor) and causes a lot of confusion.

Does that better explain things? As I mentioned in other threads, this really needs better documentation and is on my backlog of things to do.

[untoreh]

@cyphar

You should either use it with --console-socket, or set the stdio of runc create to files that you want to be linked to the container's stdio.

that is runc run -d 1>out 2>err works with terminal=false the recvtty utility works with terminal=true and --console-socket but I can't manage to daemonize it (because of stdin?), so still can't properly detach containers with terminal=true

[cyphar]

@untoreh recvtty is a PoC. If you want to daemonise everything with a terminal allocated, you need to write a program that will hold on to that fd in a way you'd like -- recvtty is effectively just a reference implementation.

[untoreh]

I used empty which does pretty much just that

[Meai1]

can you give step by step instructions on how to make this work? doesnt seem right that I cant launch a container without an external tool

[cyphar]

@Meai1

You can launch a container without recvtty -- the problem is that in order to launch a detached container that has a pty allocated requires that you have a process which runc can give the master-pty fd to (recvtty is an example tool to show you how it would work -- since how it works depends enitrely upon how your higher-level manager works). If the master-pty is closed then the container would likely be killed (or you won't be able to get output from it or send it input anymore).

We added some documentation about terminal modes here, so look at that for more detail.

[ChethanSuresh]

This is also observed in different way,

TERMINAL 1: launch container with terminal:true

host# runc run test 
# ls
bin  dev  etc lib  lib64  proc  sys

TERMINAL 2

# runc list 
ID          PID         STATUS      BUNDLE      CREATED        OWNER
test        14674       running    <bundle>   2019-04-23T04:54:19.894407185Z   root
# pstree -ps 14674 
systemd(1)───lightdm(1379)───lightdm(1696)───upstart(1862)───/usr/bin/termin(3739)───bash(4594)───sudo(4961)───su(4965)───bash(4966)───runc(14663)───sh(14674)
# kill -9 4966 
# pstree -ps 14674 
systemd(1)───lightdm(1379)───lightdm(1696)───upstart(1862)───runc(14663)───sh(14674)

• So if parent process bash(4966) is killed, then runc is attached to nearest ancestor with sub-reaper
• But the TERMINAL1 does not work anymore, just prints prompt continuously
• What if I want to exit runc calling process(runc's parent process) and get working prompt?