concourse / concourse-docker

Offical concourse/concourse Docker image.
Apache License 2.0
241 stars 153 forks source link

using a registry-image in a task fails in concourse 7.6.0 #79

Closed natto1784 closed 2 years ago

natto1784 commented 2 years ago

Similar to #76 stderr says

time="2022-02-03T08:24:16.577250855Z" level=error msg="failed to enable controllers ([cpuset cpu io memory hugetlb pids rdma])" error="failed to write subtree controllers [cpuset cpu io memory hugetlb pids rdma] to \"/sys/fs/cgroup/garden/cgroup.subtree_control\": write /sys/fs/cgroup/garden/cgroup.subtree_control: no such file or directory"
time="2022-02-03T08:24:24.862028047Z" level=info msg="starting signal loop" namespace=concourse path=/run/containerd/io.containerd.runtime.v2.task/concourse/38503fc9-0482-4038-71d8-77d6a77fd11c pid=746
time="2022-02-03T08:24:25.070572438Z" level=error msg="failed to enable controllers ([cpuset cpu io memory hugetlb pids rdma])" error="failed to write subtree controllers [cpuset cpu io memory hugetlb pids rdma] to \"/sys/fs/cgroup/cgroup.subtree_control\": write /sys/fs/cgroup/cgroup.subtree_control: operation not supported"

The problem is exactly the same as https://github.com/concourse/concourse/discussions/7953 Copying the entrypoint from #77 doesn't help either. The only difference is

-time="2022-02-03T08:24:25.070572438Z" level=error msg="failed to enable controllers ([cpuset cpu io memory hugetlb pids rdma])" error="failed to write subtree controllers [cpuset cpu io memory hugetlb pids rdma] to \"/sys/fs/cgroup/cgroup.subtree_control\": write /sys/fs/cgroup/cgroup.subtree_control: operation not supported"
+time="2022-02-03T08:24:25.070572438Z" level=error msg="failed to enable controllers ([cpuset cpu io memory hugetlb pids rdma])" error="failed to write subtree controllers [cpuset cpu io memory hugetlb pids rdma] to \"/sys/fs/cgroup/cgroup.subtree_control\": write /sys/fs/cgroup/cgroup.subtree_control: device or resource busy"

CONCOURSE_RUNTIME is set as "containerd" for the worker. Running this modified image https://github.com/robinhuiser/concourse-arm64 for my RPi

robinhuiser commented 2 years ago

Hi @natto1784!

natto1784 commented 2 years ago

@robinhuiser Thank you for responding!

which version of Concourse are you using?

7.6.0

is this happening with every registry image task?

I've only tried the busybox:latest image. The task is a simple echo hello world

is this only applicable with dcind?

Not Likely, but I have only used the docker image for concourse (https://hub.docker.com/r/rdclda/concourse) since I'm running it on a Raspberry PI 4. Please note that I'm simply running two concourse images separately as web and worker nodes, both under Nomad.

can you share the task file?

Yes!

The web UI shows this btw Concourse UI

natto1784 commented 2 years ago

Okay, this solved itself, looks like I wasted so many hours over nothing. What I did was

  1. Wipe the entire concurse-worker work directory
  2. WIpe the entire concourse DB (it was a fresh install anyway)
  3. Set "default-cgroupns-mode" to "host" in docker daemon

Then it worked! With no errors!

  1. Then I removed the "default-cgroupns-mode" setting I had set earlier

After restarting everything, it worked this time too! But the errors I mentioned in my original post are still there, I did read up on cgroups, but I'm not sure how these errors can affect me presently. At least, everything works now. Another thing to note is that I did NOT use the patch in #77 and it still worked. The patch still has the same effect that I mentioned in my original post.