hercules-ci / arion

Run docker-compose with help from Nix/NixOS
Apache License 2.0
667 stars 48 forks source link

Failed to create /init.scope control group: Read-only file system #122

Open roberth opened 3 years ago

roberth commented 3 years ago

To reproduce

  1. start a systemd-based container on NixOS 21.05
  2. observe the error message in the log
Failed to create /init.scope control group: Read-only file system
Failed to allocate manager object: Read-only file system
[!!!!!!] Failed to allocate manager object.
Exiting PID 1...

Cause

Systemd and docker have become incompatible since 21.05 and cgroupsv2.

Previously, the systemd and docker teams have disagreed about how the two should work together, so I don't expect the root cause to be resolved soon. Luckily, not the whole container ecosystem thinks about containers like docker does.

Solution

Arion >=0.1.3.0 creates containers in such a way that they do work in Podman. Podman is maintained by Red Hat, so its systemd support isn't going away anytime soon.

Start by installing podman in your NixOS configuration:

{
  virtualisation.podman.enable = true;
  virtualisation.podman.defaultNetwork.dnsname.enable = true;
  # Use your username instead of `myuser`
  users.extraUsers.myuser.extraGroups = ["podman"];
}

If you want to replace Docker by Podman entirely, use:

{
  virtualisation.podman.dockerSocket.enable = true;
  environment.systemPackages = [
     pkgs.docker-client
  ];
}

Or if you want to use them side by side, switch to Podman with:

export DOCKER_HOST=unix:///run/podman/podman.sock

and switch to local docker with:

unset DOCKER_HOST

Or add it to shell.nix for projects that need Podman.

pkgs.mkShell {  # using mkShell for example; most derivation-producing functions work the same
  # ...
  DOCKER_HOST = "unix:///run/podman/podman.sock";
}
zimbatm commented 3 years ago

If you just want things to work again before finding a permanent solution, setting systemd.enableUnifiedCgroupHierarchy = false; and rebooting should also work around the issue.

gbtb commented 1 year ago

I want to add that if you are installing podman for NixOS first time you have to reboot, because otherwise docker compat socket is not accessible with some permission errors. Perhaps this should be added to the docs as well.

Silver-Golden commented 1 year ago

I want to add that if you are installing podman for NixOS first time you have to reboot, because otherwise docker compat socket is not accessible with some permission errors. Perhaps this should be added to the docs as well.

Thank you thank you thank you, you saved me.
I have been debugging for ages why this wasnt working well and rebooting was teh solution.

deliciouslytyped commented 1 year ago

It's possible to do this rootless by setting DOCKER_HOST to /run/user/$(id -u)/podman/podman.sock .

KiaraGrouwstra commented 2 weeks ago

systemd dropping cgroup v1 seems to now make arion yield:

Legacy cgroup v1 support selected. This is no longer supported. Will proceed anyway after 30s

to address this, i tried:

podman seems to also have a --cgroups=disabled, tho i hadn't figured out yet how to set that yet thru arion.