/proc/stat is not work in cgroup v2

[hdp7891000]

Including the latest 5.0.3 version……

[mihalicyn]

Hi @hdp7891000

Couldn't you be more specific about the problem that you've met?

[hdp7891000]

Thanks. I use cgroup v2 on the host, and use docker's -v /var/lib/lxcfs/proc/stat:/proc/stat option to bind /proc/stat to the container. In the container, cat /proc/stat gets the contents of the host.

[idoyo7]

I am also using lxcfs in kubernetes with admission webhook. but I have same problem.

My cluster is working on Ubuntu 20.04 and I recently upgraded my Kuberenetes version to 1.25 and changed cgroup v1 to cgroup v2

https://github.com/ymping/lxcfs-admission-webhook

in alipine linux daemonset, I am using 5.0.3-r1 package

in this project, this code normally mounts

  - /var/lib/lxc/lxcfs/proc/cpuinfo:/proc/cpuinfo
  - /var/lib/lxc/lxcfs/proc/diskstats:/proc/diskstats
  - /var/lib/lxc/lxcfs/proc/meminfo:/proc/meminfo
  - /var/lib/lxc/lxcfs/proc/stat:/proc/stat
  - /var/lib/lxc/lxcfs/proc/swaps:/proc/swaps
  - /var/lib/lxc/lxcfs/proc/uptime:/proc/uptime
  - /var/lib/lxc/lxcfs/proc/loadavg:/proc/loadavg
  - /var/lib/lxc/lxcfs/sys/devices/system/cpu/online:/sys/devices/system/cpu/online

so I added code to mount

- /var/lib/lxcfs/proc/slabinfo:/proc/slabinfo:rw \
- /var/lib/lxcfs/sys/devices/system/cpu:/sys/devices/system/cpu:rw 

(changed cpu mount point /cpu/online to /cpu)

I could see all the cpu informations when I see /proc/stat. But I could check /proc/cpuinfo works normally working

[mihalicyn]

@hdp7891000

Couldn't you show the lxcfs logs?

@mjrodan1

I could see all the cpu informations when I see /proc/stat. But I could check /proc/cpuinfo works normally working

Couldn't you show lxcfs logs and contents of both files from inside the container?

[hdp7891000]

lxcfs doesn't print anything special.

I think read_cpuacct_usage_all does not support cgroup v2, because there is no cpuacct.usage_all or cpuacct.usage_percpu.

        /*
     * Read cpuacct.usage_all for all CPUs.
     * If the cpuacct cgroup is present, it is used to calculate the container's
     * CPU usage. If not, values from the host's /proc/stat are used.
     */
    if (read_cpuacct_usage_all(cg, cpuset, &cg_cpu_usage, &cg_cpu_usage_size) == 0) {
        if (cgroup_ops->can_use_cpuview(cgroup_ops) && opts && opts->use_cfs) {
            total_len = cpuview_proc_stat(cg, cpuset, cg_cpu_usage,
                              cg_cpu_usage_size, f,
                              d->buf, d->buflen);
            goto out;
        }
    } else {
        lxcfs_v("proc_stat_read failed to read from cpuacct, falling back to the host's /proc/stat");
    }

        //maybe need to add a cgroup v2 here
    if (!cgroup_ops->get(cgroup_ops, "cpuacct", cg, "cpuacct.usage_all", &usage_str)) {
        char *sep = " \t\n";
        char *tok;

        /* Read cpuacct.usage_percpu instead. */
        lxcfs_debug("Falling back to cpuacct.usage_percpu");
        if (!cgroup_ops->get(cgroup_ops, "cpuacct", cg, "cpuacct.usage_percpu", &usage_str))
            return -1;
        ……
        }
    } else {
        if (sscanf(usage_str, "cpu user system\n%n", &read_cnt) != 0)
            return log_error(-1, "read_cpuacct_usage_all reading first line from %s/cpuacct.usage_all failed", cg);

[mihalicyn]

it should print proc_stat_read failed to read from cpuacct, falling back to the host's /proc/stat then.

Yes, cgroup v2 lacks of the cpuacct support at all. There is no alternative for this. And this is the Linux kernel side thing. https://github.com/lxc/lxcfs/issues/538#issuecomment-1446460031

[idoyo7]

my system has two lxcfs images. Deployment's role is to webhook pods and Daemonset tries to mount /proc

this is my lxcfs pod's (daemonset) container log

/usr/bin/lxcfs --foreground --enable-loadavg --enable-cfs /var/lib/lxc/lxcfs
Running constructor lxcfs_init to reload liblxcfs
mount namespace: 5
hierarchies:
  0: fd:   6: cpuset,cpu,io,memory,hugetlb,pids,rdma,misc
Kernel supports pidfds
Kernel does not support swap accounting
api_extensions:
- cgroups
- sys_cpu_online
- proc_cpuinfo
- proc_diskstats
- proc_loadavg
- proc_meminfo
- proc_stat
- proc_swaps
- proc_uptime
- proc_slabinfo
- shared_pidns
- cpuview_daemon
- loadavg_daemon
- pidfds

this log seems quite diffrent with docker's log where does docker container's log exist? I could try getting logs from the same directory

[mihalicyn]

Hi @hdp7891000

if it's still actual, couldn't you post output of cat /proc/stat and cat /proc/cpuinfo.

I'm trying to understand if your problem is the same what we have in https://github.com/lxc/lxcfs/issues/628 or it something else.

[mihalicyn]

Hi @idoyo7

I'm not sure that I understand your question. Couldn't you describe with a bit more detail, which problem do you have with LXCFS?

[gecon]

Hi @mihalicyn,

mentioning https://github.com/lxc/lxcfs/issues/628#issuecomment-2078794943 here for completeness.

(using --enable-cfs inside the container /proc/cpuinfo is correct, but /proc/stat shows host CPUs)