There is an error in "Running a systemd enabled app" documentation.

[bear0330]

The document here https://hub.docker.com/_/centos/ says:

Running a systemd enabled app container

In order to run a container with systemd, you will need to mount the cgroups volumes from the host. Below is an example command that will run the systemd enabled httpd container created earlier.

$ docker run -ti -v /sys/fs/cgroup:/sys/fs/cgroup:ro -p 80:80 local/c7-systemd-http

Actually, the command will cause permission error because lack of --privileged option, and -ti option will hang. It looks like we should use -d option.

[shusugmt]

Hi, I want to confirm that this is truly just an documentation error?

I've been googling around for a few days looking for the answer whether it really needs --privileged flag to run systemd in container, or not. Some says no, some says no but need to give some extra options like --cap-add SYS_ADMIN --security_opt seccomp:unconfined , some says yes.

I'm currently running my systemd enabled container without --privileged option but with --cap-add SYS_ADMIN --security_opt seccomp:unconfined

[jperrin]

It's somewhat due to the documentation, and due to changes in docker since the docs were written. They do need to be updated. You can run a basic systemd enabled container without --privileged, however there are a wide array of seemingly trivial tasks that will require some additional capabilities. Running with SYS_ADMIN and seccomp:unconfined might as well be --privileged.

[shusugmt]

@jperrin Thanks for clarification. After some time digging deeper into this, I figured out that there are two dimensions related:

1. Whether systemd running on the host
   • If the host os is running systemd, we don't need --privileged nor --add-cap SYS_ADMIN. Just passing -v /sys/fs/cgroup:/sys/fs/cgroup:ro is enough.
   • If not, we need either the above option (--privileged or --add-cap SYS_ADMIN)
2. Whether seccomp option is enabled (This depends on the combination of host OS and docker-engine version)
   • If seccomp is enabled, we need --security_opt seccomp:unconfined
   • If not, we don't need anything

In my case, I was using boot2docker host which is 1. no systemd running, and 2. seccomp enabled, so I needed both --cap-add SYS_ADMIN --security_opt seccomp:unconfined.

[jamshid]

I sure wish RedHat, CentOS, and Docker could get systemd and docker to get along.

Anyway, on Docker for Mac 1.12.0-a, to avoid errors like:

[!!!!!!] Failed to mount API filesystems, freezing.
or
Failed to mount cgroup at /sys/fs/cgroup/systemd: No such file or directory

you must run with:

$ docker run -ti --cap-add SYS_ADMIN --security-opt seccomp:unconfined -p 80:80 local/c7-systemd-httpd
systemd 219 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ -LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN)
Detected virtualization docker.
Detected architecture x86-64.

Welcome to CentOS Linux 7 (Core)!

Set hostname to <e160e3b04d50>.
Initializing machine ID from random generator.
Running in a container, ignoring fstab device entry for /dev/disk/by-uuid/4a14fa4c-8462-4bcf-8b68-315f5db5d123.
[  OK  ] Reached target Swap.
...

Maybe the docs on hub.docker.com could be updated.

[shusugmt]

@jamshid Starting from docker 1.12.1, you don't need to pass --security-opt seccomp:unconfined if you add --cap-add SYS_ADMIN

see https://github.com/docker/docker/pull/25567

[jamshid]

Thanks @s2ugimot, I assumed Docker for Mac was running latest docker but it is still 1.12.0, I guess that's why I see:

$ docker run -ti --cap-add SYS_ADMIN local/c7-systemd-httpd
[!!!!!!] Failed to mount API filesystems, freezing.

But even on another system, docker 1.12.1 on Ubuntu 16 running systemd, I can't get the example local/c7-systemd-httpd image from https://hub.docker.com/_/centos/ to work. Maybe a difference in our environments is that you're running docker server on CentOS7?

This is what I believe you're saying should work (since the docker server is systemd):

# docker run -ti -v /sys/fs/cgroup:/sys/fs/cgroup:ro local/c7-systemd-httpd
[!!!!!!] Failed to mount API filesystems, freezing.

Btw these attempts (suggestions from https://hub.docker.com/_/centos/) also fail:

# docker run -ti --cap-add SYS_ADMIN --security-opt seccomp:unconfined local/c7-systemd-httpd
Failed to mount tmpfs at /run: Permission denied
Failed to mount cgroup at /sys/fs/cgroup/systemd: Permission denied
[!!!!!!] Failed to mount API filesystems, freezing.

#  docker run -ti -v /tmp/$(mktemp -d):/run -v /sys/fs/cgroup:/sys/fs/cgroup:ro local/c7-systemd-httpd
[!!!!!!] Failed to mount API filesystems, freezing.

# docker info
Containers: 33
 Running: 30
 Paused: 0
 Stopped: 3
Images: 1072
Server Version: 1.12.1
Storage Driver: overlay
 Backing Filesystem: extfs
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge null host overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: apparmor seccomp
Kernel Version: 4.4.0-34-generic
Operating System: Ubuntu 16.04.1 LTS
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 31.31 GiB
Name: 63
ID: AUQE:OS5D:5RNW:SBMA:ZMSP:M5PG:7URM:5XTE:CSN2:4JVM:5C6E:JPYG
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Labels:
 provider=generic
Insecure Registries:
 127.0.0.0/8

[reissr]

I'm having the same problem with docker 1.12.1 (under Windows 7 with Boot2Docker)

[antoinetran]

I confirm having the same issue in Docker host CentOs 7.2 and Docker 1.12.2, with container "centos:7". The docker run shows an error message "API freeze". With SYS_ADMIN OR the seccomp unconfined, the example with httpd runs fine.

It seems Docker drops some capabilities recently, which is why the docker run command might have worked before, but not now.

This error is very important: do we really have to drop security for all containers using systemd? A container with SYS_ADMIN capatabilities has the ability to escalate and gain root access to the host, am I not right? Does user namespace recent implementation prevent this? Any ways, I have seen that SYS_ADMIN allows changing the time of the host, which is quite critical.

[kbsingh]

docker-latest on CentOS 7 is 1.12.1, can you retry with that - and if the issue persists, please open a bug report at bugzilla.redhat.com/

[antoinetran]

No, docker-engine latest in CentOs 7.2 is 1.12.2-1. See below: Installed Packages Name : docker-engine Arch : x86_64 Version : 1.12.2 Release : 1.el7.centos Size : 79 M

[kbsingh]

@antoinetran CentOS does not ship a package called 'docker-engine';

[antoinetran]

You're right! I discovered the "docker-latest" from CentOs Extra repo today. This version is in conflict with docker-engine, from offical repo (Docker), so I have not tried it yet. But our project would rather use the official rpm from Docker than the one from CentOs Extra repo, so even if this work, it does not correct the bug with the official Docker. Maybe this should be a bug in Docker instead of this container?

[antoinetran]

I'm not sure this is the right place, but I create a bug in: https://bugzilla.redhat.com/show_bug.cgi?id=1392526

[antoinetran]

Hi again, I suppose noone succeeded in launching the example (httpd) with the minimal rights? Like adding volume tmpfs /tmp /var /var/run (which is not sufficient from my tests)? There must be a way to avoid the freeze and at the same time avoid adding SYS_ADMIN capabilities, which is overkill for systemd. Thank you.

[Hubbitus]

What interesting, docker 1.12 does not require --cap-add SYS_ADMIN to run, but on 1.13 it is absolutely required otherwise you got: Failed to mount API filesystems, freezing.

Why it changed? Could someone explain and point on documentation?

[antoinetran]

I'm sure 1.11 does not requires SYS_ADMIN. But if I remember correctly, 1.12 requires SYS_ADMIN.

[da115115]

Has anyone been able to run systemctl-based services in a CentOS (or even Fedora) container on a MacOS host?

[pmcdowell-okta]

I had a container that used to run fine, it suddenly stopped working. Trying to figure out what is going on myself.

Here is the image that i built that used to fun fine:

https://hub.docker.com/r/oktaadmin/oktaonprem/

[da115115]

Thanks @pmcdowell-okta! However, that Docker image does not solve the issue on MacOS for me (latest versions of MacOS and Docker for Mac, in my case). Being overkilling, I have even tried things such as:

$ docker run --tmpfs /tmp --tmpfs /run --privileged -v /sys/fs/cgroup:/sys/fs/cgroup:ro \
  --rm -it opentrep/search-travel:legacy bash
[root@4bda1a2e4615 trep]# systemctl status mariadb.service
Failed to get D-Bus connection: Operation not permitted
[root@4bda1a2e4615 trep]# exit

[frankfont]

Nothing is working for me yet using centos:7 docker image to build in an Ubuntu 16 host. Here are the details...

DOCKER RUNTIME

root@mymachine # docker --version Docker version 17.09.0-ce, build afdb6d4

UBUNTU HOST

root@mymachine # uname -r 4.10.0-38-generic

root@mymachine # systemctl --version systemd 229 +PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ -LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN

Dockerfile

FROM centos:7 ENV container docker RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in ; do [ $i == \ systemd-tmpfiles-setup.service ] || rm -f $i; done); \ rm -f /lib/systemd/system/multi-user.target.wants/;\ rm -f /etc/systemd/system/.wants/;\ rm -f /lib/systemd/system/local-fs.target.wants/; \ rm -f /lib/systemd/system/sockets.target.wants/udev; \ rm -f /lib/systemd/system/sockets.target.wants/initctl; \ rm -f /lib/systemd/system/basic.target.wants/;\ rm -f /lib/systemd/system/anaconda.target.wants/*; VOLUME [ "/sys/fs/cgroup", "/tmp", "/run", "/run/lock" ] CMD ["/usr/sbin/init"]

BUILD RESULT

root@mymachine # docker build --rm -t local/c7-systemd . Sending build context to Docker daemon 6.656kB Step 1/5 : FROM centos:7 ---> 196e0ce0c9fb Step 2/5 : ENV container docker ---> Using cache ---> 70cb6837d59e Step 3/5 : RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in ; do [ $i == systemd-tmpfiles-setup.service ] || rm -f $i; done); rm -f /lib/systemd/system/multi-user.target.wants/;rm -f /etc/systemd/system/.wants/;rm -f /lib/systemd/system/local-fs.target.wants/; rm -f /lib/systemd/system/sockets.target.wants/udev; rm -f /lib/systemd/system/sockets.target.wants/initctl; rm -f /lib/systemd/system/basic.target.wants/;rm -f /lib/systemd/system/anaconda.target.wants/*; ---> Using cache ---> b4bc090ddb4a Step 4/5 : VOLUME /sys/fs/cgroup /tmp /run /run/lock ---> Using cache ---> a569a2d98f7b Step 5/5 : CMD /usr/sbin/init ---> Using cache ---> a8b9a4f1daf7 Successfully built a8b9a4f1daf7 Successfully tagged local/c7-systemd:latest

RUN RESULT

root@mymachine # docker run -ti -v /sys/fs/cgroup:/sys/fs/cgroup:ro local/c7-systemd [!!!!!!] Failed to mount API filesystems, freezing.

Is anyone having success building a CentOS7/systemd image and running it on an Ubuntu16 host?

[lijun1234]

Finally get httpd work under centos:7 docker image. The last step on https://github.com/docker-library/docs/tree/master/centos#systemd-integration will cause error like, [!!!!!!] Failed to mount API filesystems, freezing. I use the following command to create container. docker run --privileged -dti -v /sys/fs/cgroup:/sys/fs/cgroup:ro -p 88:80 local/c7-systemd-httpd docker exec -it $(docker ps -lq) systemctl status httpd â httpd.service - The Apache HTTP Server Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled) Active: active (running) since Sun 2017-11-26 03:02:31 UTC; 2s ago Docs: man:httpd(8) man:apachectl(8) Main PID: 143 (httpd) Status: "Processing requests..." CGroup: /docker/f1191d7aae17811e0b528c0c20831d5f015bda8dad265dcc86f555e0f51a6a49/system.slice/httpd.service ââ143 /usr/sbin/httpd -DFOREGROUND ââ144 /usr/sbin/httpd -DFOREGROUND ââ145 /usr/sbin/httpd -DFOREGROUND ââ146 /usr/sbin/httpd -DFOREGROUND ââ147 /usr/sbin/httpd -DFOREGROUND ââ148 /usr/sbin/httpd -DFOREGROUND

Nov 26 03:02:31 f1191d7aae17 systemd[1]: Starting The Apache HTTP Server... Nov 26 03:02:31 f1191d7aae17 httpd[143]: AH00558: httpd: Could not reliably determine the server's fully qual...ssage Nov 26 03:02:31 f1191d7aae17 systemd[1]: Started The Apache HTTP Server. Hint: Some lines were ellipsized, use -l to show in full.

Hope this can help you.

[antoinetran]

privileged mode is too much. --cap-add SYS_ADMIN is enough.

[lijun1234]

Doesn't work to me, as below. docker run -itd --cap-add SYS_ADMIN -v /sys/fs/cgroup:/sys/fs/cgroup:ro -p 80:80 c7-systemd-httpd edea3f7d5eef0bcfe59945947a3bb1eecc1bf555ba411e03c3c1d29fffdecd1c docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES edea3f7d5eef c7-systemd-httpd "/usr/sbin/init" 6 seconds ago Up 6 seconds 0.0.0.0:80->80/tcp unruffled_wozniak docker exec -it unruffled_wozniak /bin/bash systemctl stop httpd Failed to get D-Bus connection: Operation not permitted

[antoinetran]

I think you missed tmpfs: docker run -itd --cap-add SYS_ADMIN --tmpfs /run -v /sys/fs/cgroup:/sys/fs/cgroup:ro -p 80:80 c7-systemd-httpd Also, nothing to do, but don't forget the stop signal ("stop_signal: RTMIN+3" for compose).

[antoinetran]

Let me add one thing: it is best to add environment container=docker, for systemd to know this is a container. I see this has changed its output (without it, even with tty, you won't see output in /sbin/init). Correct way of launching httpd container in the example (with official docker-ce 17.06.0): docker run -itd -e container=docker --cap-add SYS_ADMIN --tmpfs /run -v /sys/fs/cgroup:/sys/fs/cgroup:ro -p 80:80 c7-systemd-httpd

[ferdinandosimonetti]

Hello everybody I followed this thread's instructions and I was eventually able to have a running CentOS7 image with sshd enabled and various other customizations suiting my needs. Now, however, comes the hard (for me) part: I need a docker-compose.yml that enables me to run an arbitrary number of sshd-enabled CentOS7 containers on my home swarm as a docker stack. Is the addition of privileged: true tmpfs: /run environment:

• container=docker volumes:
  • /sys/fs/cgroup:/sys/fs/cgroup:ro

sufficient to get things running as desired?

[kuncao]

please refer my solution https://github.com/docker-library/docs/issues/1177

[antoinetran]

We still need SYS_ADMIN for systemd and this is normal according to their developpers. I created an issue for enhancement https://github.com/systemd/systemd/issues/8657 , but nevertheless this documentation still need to be updated.

[vigneshsolution]

Hi, I want to confirm that this is truly just an documentation error?

I've been googling around for a few days looking for the answer whether it really needs --privileged flag to run systemd in container, or not. Some says no, some says no but need to give some extra options like --cap-add SYS_ADMIN --security_opt seccomp:unconfined , some says yes.

I'm currently running my systemd enabled container without --privileged option but with --cap-add SYS_ADMIN --security_opt seccomp:unconfined

Hi, I want to confirm that this is truly just an documentation error?

I've been googling around for a few days looking for the answer whether it really needs --privileged flag to run systemd in container, or not. Some says no, some says no but need to give some extra options like --cap-add SYS_ADMIN --security_opt seccomp:unconfined , some says yes.

I'm currently running my systemd enabled container without --privileged option but with --cap-add SYS_ADMIN --security_opt seccomp:unconfined

@shusugmt

I have followed the steps which you mentioned, but while running the image it's prompting for login. I have tried by given my server credentials but no luck. Can you please guide me on this.

[root@server ~]# docker run --cap-add SYS_ADMIN -ti -e container=docker -v /sys/fs/cgroup:/sys/fs/cgroup centos7-systemd /usr/sbin/init systemd 219 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN) Detected virtualization docker. Detected architecture x86-64.

Welcome to CentOS Linux 7 (Core)!

Set hostname to <7b910f14bb69>. [ OK ] Created slice Root Slice. [ OK ] Created slice System Slice. [ OK ] Reached target Remote File Systems. [ OK ] Listening on /dev/initctl Compatibility Named Pipe. [ OK ] Created slice system-getty.slice. [ OK ] Listening on Delayed Shutdown Socket. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on Journal Socket. Starting Journal Service... [ OK ] Created slice User and Session Slice. Mounting Huge Pages File System... [ OK ] Reached target Swap. [ OK ] Reached target Slices. [ OK ] Reached target Paths. Starting Load/Save Random Seed... Starting Rebuild Hardware Database... [ OK ] Reached target Local File Systems (Pre). [ OK ] Reached target Local File Systems. Starting Rebuild Journal Catalog... [ OK ] Started Journal Service. [FAILED] Failed to mount Huge Pages File System. See 'systemctl status dev-hugepages.mount' for details. [ OK ] Started Load/Save Random Seed. [ OK ] Started Rebuild Journal Catalog. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. Starting Create Volatile Files and Directories... [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started Rebuild Hardware Database. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Permit User Sessions... [ OK ] Started D-Bus System Message Bus. Starting D-Bus System Message Bus... Starting Login Service... [ OK ] Started Permit User Sessions. Starting Cleanup of Temporary Directories... [ OK ] Started Console Getty. Starting Console Getty... [ OK ] Reached target Login Prompts. [ OK ] Started Cleanup of Temporary Directories. [ OK ] Started Login Service. [ OK ] Reached target Multi-User System. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes.

CentOS Linux 7 (Core) Kernel 3.10.0-862.11.6.el7.x86_64 on an x86_64

7b910f14bb69 login:

[antoinetran]

@viki6 I don't know what you are trying to achieve but there is nothing wrong with your log. Your systemd started correctly.

For everyone: with latest systemd 239 (but not in 219), there is a fix. It is smart enough to know we are in docker and do not try to freeze at mount (source: systemd/systemd#8657). In CentOs/Redhat, this has been backported since systemd-libs-219-57, so in 7.5.1708, but not in 7.4.1708. In Fedora 28, it works fine. This works: sudo docker run --name test --stop-signal=RTMIN+3 --tmpfs /run -v /sys/fs/cgroup:/sys/fs/cgroup:ro -e container=docker --rm -ti centos:7.5.1804 /sbin/init sudo docker run --name test--stop-signal=RTMIN+3 --tmpfs /run -v /sys/fs/cgroup:/sys/fs/cgroup:ro -e container=docker --rm -ti fedora /sbin/init

This does not work without SYS_ADMIN: sudo docker run --name test --stop-signal=RTMIN+3 --tmpfs /run -v /sys/fs/cgroup:/sys/fs/cgroup:ro -e container=docker --rm -ti centos:7.4.1708 /sbin/init