`-d` option for `enroot start` ?

[ltalirz]

docker run -d will create a new container and run it in the background.

It would be useful to either document how to achieve the equivalent for enroot (one-liner), or to add a -d option as well.

cc @matt-chan

[3XX0]

There is nothing specific to Enroot, you can background the process spawned by Enroot like any other process (e.g. setsid, nohup, systemd-run, etc)

[ltalirz]

Thanks for the fast feedback, and sorry for the slow follow-up! I'd be happy to add a short note on this in the docs, e.g. here if that's considered useful.

I also notice that the enroot list command only lists the container root filesystems created via enroot create. How would you suggest to keep track of enroot containers that have been enroot started in the background (think: docker ps)?

[flx42]

You can use enroot list -f for that.

$ enroot list -f
NAME    PID  COMM  STATE  STARTED  TIME  MNTNS  USERNS  COMMAND
centos                                                  
ubuntu  

$ enroot start centos sleep 120s &
[1] 8115

$ enroot list -f
NAME    PID   COMM   STATE  STARTED  TIME   MNTNS       USERNS      COMMAND
centos  8115  sleep  S      15:51    00:02  4026533224  4026533223  /usr/bin/coreutils --coreutils-prog-shebang=sleep /usr/bin/sleep 120s
ubuntu 

[ltalirz]

Ah perfect, thanks!

Based on the help string

 Options:
   -f, --fancy  Display more information in tabular format

I did not expect -f to show something fundamentally different, so perhaps it might be worth adding a word on this in the help string.

Also, I notice that the PID of running containers will not be shown if one uses the --root option to be remapped as root inside the container. E.g.

$ enroot start --root centos.sqsh sleep 120s &

$ enroot list -f

Is this intentional?

Edit: I'm off now; will get back to this tomorrow.

[ltalirz]

P.S. Correct me if I'm wrong but when shutting down enroot containers, I understand that the responsibility lies with the user to make sure that all processes of the container are cleaned up.

E.g. if killing the main process of a containerized application does not properly terminate its child processes, those child processes will continue running (and, in fact, after killing the main process, the child processes will start being picked up by the ps query in enroot list -f). Which processes these are is not necessarily straightforward to determine (?); I guess one needs to rely on the application developer to make sure the container exits cleanly.

This may be obvious to advanced users of enroot; just something to be aware of.

P.P.S. Just to provide one possible solution in case someone else needs this as well:

Start container

setsid enroot start $PATH_TO_IMAGE $CMD &> container.log &

Force-kill container (both the parent process and its children)

$ enroot list -f
NAME       PID      COMM  STATE  STARTED  TIME   MNTNS       USERNS      COMMAND
<unknown>  3202027  bash  Ss     10:37    00:03  4026532440  4026532439  mycmd
$ PGID=$( ps -o pgid "3202027" | grep [0-9] | tr -d ' ' )
$ kill -9 -$PGID

[flx42]

Also, I notice that the PID of running containers will not be shown if one uses the --root option to be remapped as root inside the container [...]

This is because you are using centos.sqsh, not --root so it's not extracting the sqsh file to a directory unlike when you do enroot create + enroot start

[3XX0]

Correct me if I'm wrong but when shutting down enroot containers, I understand that the responsibility lies with the user to make sure that all processes of the container are cleaned up.

That's right, Enroot is not a container per-se, merely a fancy chroot. This is by design, once Enroot has done its job and executed the process it leaves no trace on the system.

You should handle your processes the same way as bare-metal, whether it's backgrounding, reaping or listing (enroot list is just a ps helper). Usually this would involve a workload manager of some kind (e.g. systemd, Slurm, etc)