lxc / lxcfs

FUSE filesystem for LXC
https://linuxcontainers.org/lxcfs
Other
1.05k stars 250 forks source link

can't get right /proc/cpuinfo in docker #642

Closed rfyiamcool closed 6 months ago

rfyiamcool commented 6 months ago

summary

can't get expected cpuinfo in docker, still get /proc/cpuinfo of the node in container.

image

dep versions

System OS

ubuntu: 24:04

lxc version

5.0.0

lxc install method

# ubuntu

apt install lxcfs
systemctl restart lxcfs

lxc status

root@k8snode01:~/ruifengyun# systemctl status lxcfs
● lxcfs.service - FUSE filesystem for LXC
     Loaded: loaded (/lib/systemd/system/lxcfs.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2024-05-10 11:49:00 CST; 2h 26min ago
       Docs: man:lxcfs(1)
   Main PID: 508787 (lxcfs)
      Tasks: 5 (limit: 98303)
     Memory: 2.0M
        CPU: 1.932s
     CGroup: /system.slice/lxcfs.service
             └─508787 /usr/bin/lxcfs /var/lib/lxcfs

May 10 11:49:00 k8snode01 lxcfs[508787]: - proc_loadavg
May 10 11:49:00 k8snode01 lxcfs[508787]: - proc_meminfo
May 10 11:49:00 k8snode01 lxcfs[508787]: - proc_stat
May 10 11:49:00 k8snode01 lxcfs[508787]: - proc_swaps
May 10 11:49:00 k8snode01 lxcfs[508787]: - proc_uptime
May 10 11:49:00 k8snode01 lxcfs[508787]: - proc_slabinfo
May 10 11:49:00 k8snode01 lxcfs[508787]: - shared_pidns
May 10 11:49:00 k8snode01 lxcfs[508787]: - cpuview_daemon
May 10 11:49:00 k8snode01 lxcfs[508787]: - loadavg_daemon
May 10 11:49:00 k8snode01 lxcfs[508787]: - pidfds

docker version

root@k8snode01:~/ruifengyun# docker version
Client: Docker Engine - Community
 Version:           26.0.0
 API version:       1.45
 Go version:        go1.21.8
 Git commit:        2ae903e
 Built:             Wed Mar 20 15:17:48 2024
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          26.0.0
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.8
  Git commit:       8b79278
  Built:            Wed Mar 20 15:17:48 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.28
  GitCommit:        ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker run test

limit container to set memory 256m and 2 cpu core, i can get right meminfo in container, but get wrong unlimited cpuinfo. 😅 I wnat to get 2 cpu core only in container.

test1

root@k8snode01:~/ruifengyun# docker run --rm --name limited-pod -ti --memory 256m --memory-swappiness 0 --cpus=2  -v /var/lib/lxcfs/proc/cpuinfo:/proc/cpuinfo:rw  -v /var/lib/lxcfs/proc/diskstats:/proc/diskstats:rw  -v /var/lib/lxcfs/proc/meminfo:/proc/meminfo:rw  -v /var/lib/lxcfs/proc/stat:/proc/stat:rw  -v /var/lib/lxcfs/proc/swaps:/proc/swaps:rw -v /var/lib/lxcfs/proc/uptime:/proc/uptime:rw  ubuntu:24.04 /bin/bash
WARNING: Your kernel does not support memory swappiness capabilities or the cgroup is not mounted. Memory swappiness discarded.
root@ad9264628ebe:/#
root@ad9264628ebe:/# free -m
               total        used        free      shared  buff/cache   available
Mem:             256           1         254           0           0         254
Swap:              0           0           0
root@ad9264628ebe:/#
root@ad9264628ebe:/# cat /proc/cpuinfo |grep processor|wc -l
80
root@ad9264628ebe:/#

test2

root@k8snode01:~/ruifengyun# docker run --rm --name limited-pod -ti --memory 256m --memory-swappiness 0 --cpu-period=100000 --cpu-quota=200000 -v /var/lib/lxcfs/proc/cpuinfo:/proc/cpuinfo:rw -v /var/lib/lxcfs/proc/diskstats:/proc/diskstats:rw -v /var/lib/lxcfs/proc/meminfo:/proc/meminfo:rw  -v /var/lib/lxcfs/proc/stat:/proc/stat:rw  -v /var/lib/lxcfs/proc/swaps:/proc/swaps:rw  -v /var/lib/lxcfs/proc/uptime:/proc/uptime:rw  ubuntu:18.04 /bin/bash

...

root@125b4f278e96:/# free -m
              total        used        free      shared  buff/cache   available
Mem:            256           1         254           0           0         254
Swap:             0           0           0

root@125b4f278e96:/# cat /proc/cpuinfo|grep -i processor|wc -l
80
stgraber commented 6 months ago

That's normal, --cpus configures a CFS quota, not a cpuset, so by default lxcfs will not do anything with that. If you had used --cpuset-cpus instead, it'd have worked.

Starting LXCFS with --enable-cfs should also make things behave better in your case.