opencontainers / runc

CLI tool for spawning and running containers according to the OCI specification
https://www.opencontainers.org/
Apache License 2.0
11.89k stars 2.11k forks source link

runc update will clear cpu burst value #4210

Closed lifubang closed 7 months ago

lifubang commented 8 months ago
  1. runc update will clear cpu burst value

    root@acmcoder:/home/acmcoder/ubuntutest# cat /sys/fs/cgroup/cpu/user.slice/test/cpu.cfs_burst_us
    0
    root@acmcoder:/home/acmcoder/ubuntutest#
    root@acmcoder:/home/acmcoder/ubuntutest# runc update --cpu-burst=10000 test
    root@acmcoder:/home/acmcoder/ubuntutest# cat /sys/fs/cgroup/cpu/user.slice/test/cpu.cfs_burst_us
    10000
    root@acmcoder:/home/acmcoder/ubuntutest# runc update --memory=100M test
    root@acmcoder:/home/acmcoder/ubuntutest# cat /sys/fs/cgroup/cpu/user.slice/test/cpu.cfs_burst_us
    0
  2. Error for container update for the kernel not support cpu burst When I'm testing #4188 in ubuntu 20.04 with cgroupv2, I got some errors:

    ✗ update cgroup v1/v2 common limits
    (in test file tests/integration/update.bats, line 74)
     `[ "$status" -eq 0 ]' failed
    runc spec (status=0):
    
    runc run -d --console-socket /tmp/bats-run-JaQGX1/runc.L3uZyo/tty/sock test_update (status=0):
    
    cat: /sys/fs/cgroup/runc-cgroups-integration-test/test-cgroup-16459/memory.swap.max: No such file or directory
    current 33554432 !? 33554432
    current 25165824 !? 25165824
    current 20 !? 20
    runc update test_update --cpuset-cpus 1 (status=1):
    time="2024-03-03T07:53:10+08:00" level=warning msg="Setting back cgroup configs failed due to error: open /sys/fs/cgroup/runc-cgroups-integration-test/test-cgroup-16459/cpu.max.burst: no such file or directory, your state.json and actual configs might be inconsistent."
    time="2024-03-03T07:53:10+08:00" level=error msg="open /sys/fs/cgroup/runc-cgroups-integration-test/test-cgroup-16459/cpu.max.burst: no such file or directory"
root@iZj6ccy7ct4rpjdodcohzlZ:~/ubuntu# uname -a
Linux iZj6ccy7ct4rpjdodcohzlZ 5.4.0-170-generic #188-Ubuntu SMP Wed Jan 10 09:51:01 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
root@iZj6ccy7ct4rpjdodcohzlZ:~/ubuntu# ls /sys/fs/cgroup/user.slice/user-0.slice/
cgroup.controllers      cpu.max                cpu.weight           memory.low        rdma.max
cgroup.events           cpu.pressure           cpu.weight.nice      memory.max        session-104.scope
cgroup.freeze           cpuset.cpus            io.max               memory.min        session-106.scope
cgroup.max.depth        cpuset.cpus.effective  io.pressure          memory.oom.group  session-109.scope
cgroup.max.descendants  cpuset.cpus.partition  io.stat              memory.pressure   test
cgroup.procs            cpuset.mems            io.weight            memory.stat       test1
cgroup.stat             cpuset.mems.effective  memory.current       pids.current      user@0.service
cgroup.subtree_control  cpu.stat               memory.events        pids.events       user-runtime-dir@0.service
cgroup.threads          cpu.uclamp.max         memory.events.local  pids.max
cgroup.type             cpu.uclamp.min         memory.high          rdma.current
lifubang commented 8 months ago

I'll add a testcase for this issue later.

rata commented 7 months ago

The ubuntu issue you mention seems the same that breaks containerd tests with the new runc: https://github.com/containerd/containerd/actions/runs/8540558806/job/23398018642?pr=9313:

=== FAIL: integration/client TestTaskUpdate (0.23s)
    log_hook.go:47: time="2024-04-03T14:43:18.318748080Z" level=debug msg="remote introspection plugin filters" func="proxy.(*introspectionRemote).Plugins" file="/home/runner/actions-runner/_work/containerd/containerd/core/introspection/proxy/remote.go:57" filters="[type==io.containerd.snapshotter.v1, id==overlayfs]" testcase=TestTaskUpdate
    container_linux_test.go:137: runc did not terminate successfully: exit status 1: openat2 /sys/fs/cgroup/testing/TestTaskUpdate/cpu.max.burst: no such file or directory
        : unknown

@lifubang do you know already what the fix should/might look like? Or did you find a wall as @kolyshkin with his PR?

lifubang commented 7 months ago

Yes, it’s easy to fix this issue, but we found some other issues for ‘runc update’, we want to fix all of these issues in one PR, but it’s too difficult, I think maybe we should fix them step by step.