Unable to run systemd services on Docker Desktop 4.3.0

madrover commented 2 years ago

[x] I have tried with the latest version of Docker Desktop
[x] I have tried disabling enabled experimental features
[ ] I have uploaded Diagnostics
Diagnostics ID:

Expected behavior

Systemd based services in containers should start, as they have always been doing in x86 computers

Actual behavior

Systemd based services in containers do not start on M1 / Silicon computers.

Information

Hi

I've recently been given a new M1 MBP laptop where I'm trying to run a docker image with systemd that we use as part of our CI build, but I'm not able to start it successfully. We have been building and running these containers in x86 MBP for years with success, so this seems to a platform specific issue with the new M1.

macOS Version: Monterey 12.0.1
Intel chip or Apple chip: Apple chip
Docker Desktop Version: 4.3.0 (71786)

Steps to reproduce the behavior

Following the instructions at https://hub.docker.com/_/centos I've created a Centos Systemd ready container with the following Dockerfile:

FROM centos:7
ENV container docker
RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == \
systemd-tmpfiles-setup.service ] || rm -f $i; done); \
rm -f /lib/systemd/system/multi-user.target.wants/*;\
rm -f /etc/systemd/system/*.wants/*;\
rm -f /lib/systemd/system/local-fs.target.wants/*; \
rm -f /lib/systemd/system/sockets.target.wants/*udev*; \
rm -f /lib/systemd/system/sockets.target.wants/*initctl*; \
rm -f /lib/systemd/system/basic.target.wants/*;\
rm -f /lib/systemd/system/anaconda.target.wants/*;
VOLUME [ "/sys/fs/cgroup" ]
CMD ["/usr/sbin/init"]

However, when running it I get the following error:

$ docker run --rm  -ti --privileged -v /sys/fs/cgroup:/sys/fs/cgroup local/c7-systemd
Failed to insert module 'autofs4'
Failed to mount cgroup at /sys/fs/cgroup/systemd: Operation not permitted
systemd 219 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN)
Detected virtualization docker.
Detected architecture arm64.

Welcome to CentOS Linux 7 (AltArch)!

Set hostname to <6870484659ce>.
Initializing machine ID from random generator.
Cannot determine cgroup we are running in: No such file or directory
Failed to allocate manager object: No such file or directory
[!!!!!!] Failed to allocate manager object, freezing.

I've tried to use the DOCKER_DEFAULT_PLATFORM=linux/amd64 environment variable as well, but the output and behavior is mostly the same.

$ docker run --rm  -ti --privileged -v /sys/fs/cgroup:/sys/fs/cgroup local/c7-systemd
Failed to insert module 'autofs4'
Failed to mount cgroup at /sys/fs/cgroup/systemd: Operation not permitted
systemd 219 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN)
Detected virtualization docker.
Detected architecture x86-64.

Welcome to CentOS Linux 7 (Core)!

Set hostname to <a2561af401fb>.
Initializing machine ID from random generator.

Failed to configure loopback device: Connection timed out
Cannot determine cgroup we are running in: No such file or directory
Failed to allocate manager object: No such file or directory
[!!!!!!] Failed to allocate manager object, freezing.

Some unsuccessful attempts I've done:

different combinations of the privileged setting and /sys/fs/cgroupmounts
adding tmpfs settings, as suggested at https://stackoverflow.com/questions/36617368/docker-centos-7-with-systemctl-failed-to-mount-tmpfs-cgroup
tweak the cgroups settings "exec-opts": ["native.cgroupdriver=systemd"], "cgroup-parent": "docker.slice", as suggested at https://serverfault.com/questions/1053187/systemd-fails-to-run-in-a-docker-container-when-using-cgroupv2-cgroupns-priva

Is there anything that can be done to overcome this issue?

Thanks!

wpwood commented 2 years ago

I don't think that this is an M1 issue. I see something similar on an Intel Macbook when trying to run the official Ubuntu:18.04 image on 4.3.0.

From my docker compose log:

heimdall_api          | Failed to insert module 'autofs4': No such file or directory
heimdall_api          | systemd 237 running in system mode. (+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN2 +IDN -PCRE2 default-hierarchy=hybrid)
heimdall_api          | Detected virtualization docker.
heimdall_api          | Detected architecture x86-64.
heimdall_api          |
heimdall_api          | Welcome to Ubuntu 18.04.6 LTS!
heimdall_api          |
heimdall_api          | Set hostname to <28176e1eef6a>.
heimdall_api          | Failed to create /init.scope control group: Read-only file system
heimdall_api          | Failed to allocate manager object: Read-only file system
heimdall_api          | [!!!!!!] Failed to allocate manager object, freezing.
heimdall_api          | Freezing execution.

Dropping back to Docker Desktop 4.2.0 fixes the issue. I assume that it's related to the note in the 4.3.0 release notes:

Docker Desktop now uses cgroupv2. If you need to run systemd in a container then:
* Ensure your version of systemd supports cgroupv2. It must be at least systemd 247. Consider upgrading any centos:7 images to centos:8.
* Containers running systemd need the following options: --privileged --cgroupns=host -v /sys/fs/cgroup:/sys/fs/cgroup:rw.

If so, though, it's going to break a bunch of people on older images.

madrover commented 2 years ago

Good catch @wpwood! That new Docker Desktop version with the behavior change regarding systemd has been released exactly the same day I had my new M1 delivered. I was wondering why I wasn't seeing more reports about this matter, now I understand... hehehe

I haven't done much testing, but I've downgraded my M1 MBP Docker Desktop to 4.2.0 and systemd services seem correct when using an aarch64 and amd64 centos:7 containers, albeit considerably slower in the second (1'' vs 30'' aprox...). Next week I'll have a go on my Intel MBP and see how it behaves.

The requirement of systemd-247 seems quite steep. At the time of writing not even Centos 8 stream has it, being systemd-239-51 its current version, see https://pkgs.org/download/systemd. So, yeah, this is going to have a noticeable impact.

elishagreenwald commented 2 years ago

The requirement of systemd-247 seems quite steep. At the time of writing not even Centos 8 stream has it, being systemd-239-51 its current version, see https://pkgs.org/download/systemd. So, yeah, this is going to have a noticeable impact.

+1 to this. Is there any chance you'd consider an option which version of cgroups to use so you don't break backwards compatibility? I think some folks who are using systemd with centos7 will appreciate it although who runs systemd in docker containers?!? ✋ I do, lol.

djs55 commented 2 years ago

Thanks for the report. It looks like the release note recommending 247 is overly pessimistic (perhaps relevant changes were backported?)

Here's a Dockerfile I tried (on Windows but I think Mac should be the same)

FROM centos:7
ENV container docker
RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == systemd-tmpfiles-setup.service ] || rm -f $i; done); \
rm -f /lib/systemd/system/multi-user.target.wants/*;\
rm -f /etc/systemd/system/*.wants/*;\
rm -f /lib/systemd/system/local-fs.target.wants/*; \
rm -f /lib/systemd/system/sockets.target.wants/*udev*; \
rm -f /lib/systemd/system/sockets.target.wants/*initctl*; \
rm -f /lib/systemd/system/basic.target.wants/*;\
rm -f /lib/systemd/system/anaconda.target.wants/*;
RUN yum -y install httpd; yum clean all; systemctl enable httpd.service
EXPOSE 80
CMD ["/usr/sbin/init"]

and here's what I ran:

PS ...> docker build -t centos-7-test .
[+] Building 0.1s (7/7) FINISHED
 => [internal] load build definition from Dockerfile                                                                                                                                                                            0.0s
 => => transferring dockerfile: 32B                                                                                                                                                                                             0.0s
 => [internal] load .dockerignore                                                                                                                                                                                               0.0s
 => => transferring context: 2B                                                                                                                                                                                                 0.0s
 => [internal] load metadata for docker.io/library/centos:7                                                                                                                                                                     0.0s
 => [1/3] FROM docker.io/library/centos:7                                                                                                                                                                                       0.0s
 => CACHED [2/3] RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == systemd-tmpfiles-setup.service ] || rm -f $i; done); rm -f /lib/systemd/system/multi-user.target.wants/*;rm -f /etc/systemd/system/  0.0s
 => CACHED [3/3] RUN yum -y install httpd; yum clean all; systemctl enable httpd.service                                                                                                                                        0.0s
 => exporting to image                                                                                                                                                                                                          0.0s
 => => exporting layers                                                                                                                                                                                                         0.0s
 => => writing image sha256:0490abd5374172376e312e836f0e53447648ba8e3191c7bb167e9397d0592ae8                                                                                                                                    0.0s
 => => naming to docker.io/library/centos-7-test                                                                                                                                                                                0.0s

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them

PS ...> docker run --rm --name systemd --privileged -v /sys/fs/cgroup:/sys/fs/cgroup:rw --cgroupns=host -p 8080:80 -it centos-7-test
systemd 219 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN)
Detected virtualization docker.
Detected architecture x86-64.

Welcome to CentOS Linux 7 (Core)!

Set hostname to <6fc030c2ff3f>.
Initializing machine ID from random generator.
[  OK  ] Reached target Paths.
[  OK  ] Reached target Swap.
[  OK  ] Reached target Local File Systems.
[  OK  ] Created slice Root Slice.
[  OK  ] Listening on Delayed Shutdown Socket.
[  OK  ] Listening on Journal Socket.
[  OK  ] Created slice System Slice.
[  OK  ] Reached target Slices.
         Starting Create Volatile Files and Directories...
         Starting Journal Service...
[  OK  ] Started Create Volatile Files and Directories.
[ INFO ] Update UTMP about System Boot/Shutdown is not active.
[DEPEND] Dependency failed for Update UTMP about System Runlevel Changes.
Job systemd-update-utmp-runlevel.service/start failed with result 'dependency'.
[  OK  ] Started Journal Service.
[  OK  ] Reached target System Initialization.
[  OK  ] Started Daily Cleanup of Temporary Directories.
[  OK  ] Reached target Timers.
[  OK  ] Listening on D-Bus System Message Bus Socket.
[  OK  ] Reached target Sockets.
[  OK  ] Reached target Basic System.
         Starting The Apache HTTP Server...
         Starting Cleanup of Temporary Directories...
[  OK  ] Started Cleanup of Temporary Directories.
[  OK  ] Started The Apache HTTP Server.
[  OK  ] Reached target Multi-User System.

I was able to open localhost:8080 in my browser.

Could you try docker pull centos:7? It might be that patches were backported and the systemd version bumped in the latest image.

For reference I'm using

PS ...> docker run -it centos:7
[root@43b9a4866ece /]# rpm -qi systemd
Name        : systemd
Version     : 219
Release     : 78.el7
Architecture: x86_64

djs55 commented 2 years ago

Looking more closely at your setup compared to mine, I think my command-line arguments are different:

docker run --privileged -v /sys/fs/cgroup:/sys/fs/cgroup

vs

docker run --privileged -v /sys/fs/cgroup:/sys/fs/cgroup:rw --cgroupns=host

madrover commented 2 years ago

Many thanks for your input @djs55, but I'm afraid it's not working for me, either on my arm64 nor x86 laptops. I'm using the same Dockerfile and the output on both laptops is the same:

$ docker build --no-cache -t centos-7-test .
[+] Building 48.5s (7/7) FINISHED
 => [internal] load build definition from Dockerfile                                                                                                                                                                                    0.1s
 => => transferring dockerfile: 674B                                                                                                                                                                                                    0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                       0.1s
 => => transferring context: 2B                                                                                                                                                                                                         0.0s
 => [internal] load metadata for docker.io/library/centos:7                                                                                                                                                                             0.0s
 => CACHED [1/3] FROM docker.io/library/centos:7                                                                                                                                                                                        0.0s
 => [2/3] RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == systemd-tmpfiles-setup.service ] || rm -f $i; done); rm -f /lib/systemd/system/multi-user.target.wants/*;rm -f /etc/systemd/system/*.wants/*;rm -f  0.5s
 => [3/3] RUN yum -y install httpd; yum clean all; systemctl enable httpd.service                                                                                                                                                      47.2s
 => exporting to image                                                                                                                                                                                                                  0.6s
 => => exporting layers                                                                                                                                                                                                                 0.5s
 => => writing image sha256:1a5df5220c15b54d9cfef8bbf85f499efba29616b0db7734ea1c805c00f80579                                                                                                                                            0.0s
 => => naming to docker.io/library/centos-7-test                                                                                                                                                                                        0.0s
$  docker run --rm --name systemd --privileged -v /sys/fs/cgroup:/sys/fs/cgroup:rw --cgroupns=host -p 8080:80 -it centos-7-test
Failed to insert module 'autofs4'
Failed to create symlink /sys/fs/cgroup/net_cls: Operation not permitted

On the other hand, I've tested the same with the same Dockerfile but using centos:8 and it works flawlessly. I guess the key difference is the usage of Cgroups v2, which does not seem to be enabled by default on Centos 7 but it does on Centos 8, see https://www.redhat.com/en/blog/world-domination-cgroups-rhel-8-welcome-cgroups-v2.

Given that Centos 7 is still a supported and widely used platform, I would suggest to make the Cgroups V2 requirement in Docker for Mac optional. I don't have any data to back my assumption, but I would say that Centos 7 is still more prevalent than Centos 8 right now, specially with the the Centos license change which EOLs Centos 8 in a few weeks.

I'll downgrade Docker Desktop to 4.2.0 for now...

wbaweto commented 2 years ago

you can see docker desktop 4.3.0 release notes: https://docs.docker.com/desktop/mac/release-notes/

Docker Desktop now uses cgroupv2. If you need to run systemd in a container then:

Ensure your version of systemd supports cgroupv2. It must be at least systemd 247. Consider upgrading any centos:7 images to centos:8

Containers running systemd need the following options: --privileged --cgroupns=host -v /sys/fs/cgroup:/sys/fs/cgroup:rw.

junoteam commented 2 years ago

If anybody interested how to run Ubuntu 22:04 Docker with systemd inside: https://github.com/rofrano/vagrant-docker-provider/issues/5

carlosabalde commented 2 years ago

Temporarily downgrading to 4.2.0 here too. Is there any chance for that option to decide which version of cgroups to use?

carlosabalde commented 2 years ago

I've shared a few notes in https://github.com/systemd/systemd/issues/19760#issuecomment-992675467 that I think could be useful to other people landing in this case.

jamshid commented 2 years ago

Just to be clear before this regression I could run systemd in a centos:7 container without --privileged and /sbin/init started fine -- systemctl status and restart worked fine. I just had to add this docker-compose-override.yml with these additional settings for my container:

  myapp:
     # Running myapp service on centos7 with systemd is a PAIN
    cap_add:
      - SYS_ADMIN
    tmpfs: /run
    volumes:
       - /sys/fs/cgroup:/sys/fs/cgroup:ro
    environment:
      - container=docker

While I know it's not good practice to run systemd in a container it's important for being able to simulate a "regular" environment using containers e.g. to run QA scripts that use systemctl. I believe podman advertises systemd compatibility, seems important to maintain parity.

djs55 commented 2 years ago

As an experiment, could you try using this development build: https://desktop-stage.docker.com/mac/main/arm64/72295/Docker.dmg You will need to edit ~/Library/Group\ Containers/group.com.docker/settings.json to add

"legacyCgroupv1": true,

oguizol commented 2 years ago

I managed to make it work with the build for amd64 and with the settings update. Thanks!

c-hoeger commented 2 years ago

Would you mind providing such a dev snapshot build for x86_64 as well? Having the same problem now and only a downgrade to 4.2.0 solved my problem.

djs55 commented 2 years ago

@c-hoeger sorry I should have mentioned: you can replace arm64 with amd64 in the URL to find the matching x86_64 build, i.e. https://desktop-stage.docker.com/mac/main/amd64/72295/Docker.dmg

carlosabalde commented 2 years ago

@djs55: I've tried your development build for amd64 using legacyCgroupv1: true and I confirm it allows me to continue using RHEL 7 and Xenial containers running systemd without any changes (i.e. using same options a before: --privileged and -v /sys/fs/cgroup:/sys/fs/cgroup:ro). Others distros like Ubuntu Bionic, Debian Buster or RHEL 8 work fine too, but they didn't experience any issues with Docker Desktop >= 4.3.0 (even all of them ship systemd versions older than 247).

c-hoeger commented 2 years ago

@c-hoeger sorry I should have mentioned: you can replace arm64 with amd64 in the URL to find the matching x86_64 build, i.e. https://desktop-stage.docker.com/mac/main/amd64/72295/Docker.dmg

@djs55 thank you for the link. That version with legacyCgroupv1 set to true works for me as well! Will that be part of the next release (4.4.x)?

c-hoeger commented 2 years ago

And what about changing the issue summary to e.g. Unable to run systemd services with docker desktop 4.3.1 or later, since it isn't related to Apple M1?

madrover commented 2 years ago

I can confirm that it's working for me as well using the tests described at https://github.com/docker/for-mac/issues/6073#issuecomment-989890965 with both arm64 and amd64 containers. Many thanks @djs55 !

I agree with you, @c-hoeger. I've been thinking the same as well. Will do shortly :)

elishagreenwald commented 2 years ago

I can confirm the development build is working for me as well, thank you! We're using centos7. If you are going to roll out the option to choose, I'm curious what approach you'll take. Some of my questions/thoughts...

1) I'm hoping once you release this, it will be super simple to use old cgroups (at least a checkbox in the Docker Desktop UI) and not have to manually change the file. 2) What would would be the default v1 or v2? 3) From a user standpoint it would be really great if backwards compatibility was ensured at least for a certain amount of time. Like we will keep the option to use v1 around for x amount of time. Ideally backwards compatibility would always be available but I'm assuming you'd like to use v2 for reasons and therefore want to encourage adoption. 4) What's the timeline for a stable release with this functionality?

Thanks again for rolling out the option!

yamt commented 2 years ago

@c-hoeger sorry I should have mentioned: you can replace arm64 with amd64 in the URL to find the matching x86_64 build, i.e. https://desktop-stage.docker.com/mac/main/amd64/72295/Docker.dmg

since 4.3.0, i have had a trouble with docker:stable-dind, which doesn't seem to support cgroupv2. it seems working well with legacyCgroupv1.

fisher-he commented 2 years ago

@djs55 Thanks, it works for me after downloading the desktop and changing the settings.

c-hoeger commented 2 years ago

@c-hoeger sorry I should have mentioned: you can replace arm64 with amd64 in the URL to find the matching x86_64 build, i.e. https://desktop-stage.docker.com/mac/main/amd64/72295/Docker.dmg

@djs55 thank you for the link. That version with legacyCgroupv1 set to true works for me as well! Will that be part of the next release (4.4.x)?

Can we get this on the roadmap?

fisehara commented 2 years ago

@djs55 Thanks, it works for me after downloading the desktop and changing the settings.

kyoshiro commented 2 years ago

Great! I came across this issue and the additional setting as mentioned by @djs55 works for me too!

As an experiment, could you try using this development build: https://desktop-stage.docker.com/mac/main/arm64/72295/Docker.dmg You will need to edit ~/Library/Group\ Containers/group.com.docker/settings.json to add
"legacyCgroupv1": true,

ddbrodsky commented 2 years ago

I have just updated to Docker Desktop 4.4.2 and saw similar issues with:

setting "deprecatedCgroupsv1": true, in

/Users/XXXXXX/Library/Group\ Containers/group.com.docker/settings.json

and restarting docker. But then I found it had somehow populated the following:

"deprecatedCgroupv1": false, -- notice the missing s in groups. The changing the value to true makes it work properly. Can you please update the release notes with the correct key and location of the settings.json?

I feel that the file: /Users/XXXXXX/Library/Group\ Containers/group.com.docker/settings.json gets overwritten in some cases by docker but not absolutely sure about it.

Thanks!

elishagreenwald commented 2 years ago

Disappointed to see the release notes saying:

Added a deprecated option to settings.json: "deprecatedCgroupsv1": true, which switches the Linux environment back to cgroups v1. **This option will be removed in future releases**. If your software requires cgroups v1, you must update it to be compatible with cgroups v2.

I understand you're trying to push a (hopefully) better implementation but when breaking backwards compatibility for users I think it would be appreciated if you would communicate a) a very specific deadline for the breaking change b) and support backwards compatibility for as long as possible within reason (in a very used software like Docker I would think at least a year is reasonable but I can be persuaded)

Forcing a quick cgroup change will impact work our org has done. Our org are paid docker users but even if not, I feel like when releasing new stuff we should try to give users as much flexibility as possible.

Would you consider pushing the final deadline to a year from now or later and publishing an actual date/release when that would happen please? Thank you!

aem12f commented 2 years ago

Agreed, considering one of the options required --cgroupns=host doesn't seem to be available for docker-compose.

Disappointed to see the release notes saying:

Added a deprecated option to settings.json: "deprecatedCgroupsv1": true, which switches the Linux environment back to cgroups v1. **This option will be removed in future releases**. If your software requires cgroups v1, you must update it to be compatible with cgroups v2.

I understand you're trying to push a (hopefully) better implementation but when breaking backwards compatibility for users I think it would be appreciated if you would communicate a) a very specific deadline for the breaking change b) and support backwards compatibility for as long as possible within reason (in a very used software like Docker I would think at least a year is reasonable but I can be persuaded)

Forcing a quick cgroup change will impact work our org has done. Our org are paid docker users but even if not, I feel like when releasing new stuff we should try to give users as much flexibility as possible.

Would you consider pushing the final deadline to a year from now or later and publishing an actual date/release when that would happen please? Thank you!

quacktacular commented 2 years ago

Agree with the sentiment above. This communication and mitigation for this change have been handled poorly so far.

madrover commented 2 years ago

Since Docker 4.4.2 has been solved the current issue I will proceed to close it.

That said, I wholeheartedly agree about the above comments as well. Given the impact this issue has on the Docker community, I'd vouch to keep this capability to be available while the major distributions (eg Centos 7) that needs this functionality are still supported.

gtskaushik commented 2 years ago

@madrover I think there is a typo. The setting is actually

"deprecatedCgroupv1": true But in documentation and in the comments in this issue, it is mentioned as "deprecatedCgroupsv1": true

RulerOf commented 2 years ago

Figured I'd share since I took the time to write this. Here's a fix for MacOS you can paste into your terminal:

# Stop running Docker
test -z "$(docker ps -q 2>/dev/null)" && osascript -e 'quit app "Docker"'
# Install jq and moreutils so we can merge into the existing json file
brew install jq moreutils
# Add the needed cgroup config to docker settings.json
echo '{"deprecatedCgroupv1": true}' | \
  jq -s '.[0] * .[1]' ~/Library/Group\ Containers/group.com.docker/settings.json - | \
  sponge ~/Library/Group\ Containers/group.com.docker/settings.json
# Restart docker desktop
open --background -a Docker

jamshid commented 2 years ago

I can't get the setting "deprecatedCgroupv1": true, or "deprecatedCgroupsv1": true, to stay in ~/Library/Group Containers/group.com.docker/settings.json. That file gets overwritten at startup.

Ok thanks @RulerOf that script seems to work, I don't know why. Anyone know what's going on with settings.json? It doesn't seem like all that should be necessary every time you restart Docker for Mac.

djs55 commented 2 years ago

@elishagreenwald @aem12f @quacktacular and all: sorry for the typo in the release notes -- I've made a PR to correct it here: https://github.com/docker/docker.github.io/pull/14166

I've also softened the language around upgrading containers to v2. I expect the downgrade of Docker to v1 will continue working (since the Docker engine itself is well-tested there), with 2 caveats:

since Linux kernel development focuses on v2, it's possible that bugs in v1 won't get fixed, however this is probably more of a issue for production environments rather than development.
it's possible that future Docker features will depend on v2, but current features should continue to work in v1.

I hope that sounds more reasonable!

@jamshid are you still seeing the deprecatedCgroupv1 setting revert on application restart?

jamshid commented 2 years ago

Thanks @djs55 correct I could not get the setting to stick. There's definitely something buggy there since @RulerOf had to make a script.

I just tried adding "deprecatedCgroupv1": true, to ~/Library/Group Containers/group.com.docker/settings.json and did Docker menu => Restart. The setting is still in the file but docker info reports Cgroup Driver: cgroupfs Cgroup Version: 2.

And when I Quit docker and Reopen it, the setting is removed from settings.json. Hopefully you see that?

Running the script above does add "deprecatedCgroupv1": true but same output from docker info: Cgroup Version: 2. Is that expected?

Btw I'm trying to switch to cgroup V1 because I'm hoping it allows systemd in a container to work without --cgroupns=host, which docker compose does not yet support (https://github.com/docker/compose/issues/8167).

dannystaple commented 2 years ago

Figured I'd share since I took the time to write this. Here's a fix for MacOS you can paste into your terminal:

# Stop running Docker
test -z "$(docker ps -q 2>/dev/null)" && osascript -e 'quit app "Docker"'
# Install jq and moreutils so we can merge into the existing json file
brew install jq moreutils
# Add the needed cgroup config to docker settings.json
echo '{"deprecatedCgroupv1": true}' | \
  jq -s '.[0] * .[1]' ~/Library/Group\ Containers/group.com.docker/settings.json - | \
  sponge ~/Library/Group\ Containers/group.com.docker/settings.json
# Restart docker desktop
open --background -a Docker

This whole thread turned out to be the key to concourse CI no longer working locally on a Mac. There are updates in the offing for concourse, but given that the next layer is possibly DinD stuff containers, and those may still not be on groups v2 yet, then using the script to set it is the best fix until these other projects release updated containers.

aem12f commented 2 years ago

I'm not sure who needs to hear this, whether it's here or on https://github.com/docker/compose/issues/8167, but I was able to successfully get systemd processes working on ubuntu 22.04 image which comes standard with systemd 249 using docker-compose on the latest version of docker desktop (4.5.0 as i'm on a mac). Ensure cgroup vol is rw and priv is true. the sentiment has been that we needed cgroupns=host for this to work, but it has not been created as a valid flag for docker-compose. My current docker-compose setup along with relative volume paths was proving to be difficult to migrate to a docker run command. Maybe this could help someone

- /sys/fs/cgroup:/sys/fs/cgroup:rw privileged: true

stebujohny commented 2 years ago

This may help, just upgrade the systemd to the latest version. I added these extra lines in my Dockerfile. No need to downgrade Docker desktop. Works well in my MacOS.

RUN yum -y install  wget

RUN wget --no-check-certificate https://copr.fedorainfracloud.org/coprs/jsynacek/systemd-backports-for-centos-7/repo/epel-7/jsynacek-systemd-backports-for-centos-7-epel-7.repo -O /etc/yum.repos.d/jsynacek-systemd-centos-7.repo

RUN yum -y update systemd ; yum clean all

then run container,

docker run -ti --privileged --cgroupns=host -v /sys/fs/cgroup:/sys/fs/cgroup:rw --name $DOCKER_CONTAINER_NAME -d -p 6600:22 local/test-centos7

docker-robott commented 2 years ago

Closed issues are locked after 30 days of inactivity. This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. /lifecycle locked

docker / for-mac