使用Lxcfs和kubernetes Admission Webhook实现对容器资源可视化隔离

xigang commented 5 years ago

https://xigang.github.io/2019/11/09/lxcfs-admission-webhook/

wavezhang commented 4 years ago

我手动重启lxcfs进程之后，执行挂载命令

nsenter --target $PID --mount --  mount -B "$LXCFS/proc/$file" "/proc/$file"

提示

mount: /proc/meminfo: mount point does not exist.

是 docker/runc 版本有问题吗?

root@localhost:~# docker version
Client: Docker Engine - Community
 Version:           19.03.4
 API version:       1.40
 Go version:        go1.12.10
 Git commit:        9013bf583a
 Built:             Fri Oct 18 15:53:51 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.4
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.10
  Git commit:       9013bf583a
  Built:            Fri Oct 18 15:52:23 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.10
  GitCommit:        b34a5c8af56e510852c35414db4c1f4fa6172339
 runc:
  Version:          1.0.0-rc8+dev
  GitCommit:        3e425f80a8c931f88e6d94a8c831b9d5aa481657
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

xigang commented 4 years ago

@wavezhang 挂载的propagation是否指定为slave或者rslave, 看下这个issue应该就能解决你的问题了 https://github.com/lxc/lxcfs/issues/193

wavezhang commented 4 years ago

多谢 /抱拳

fjibj commented 2 years ago

我采用如下方式启动容器 docker run -dt -m 5g \ -v=/sys/fs/cgroup:/sys/fs/cgroup:ro \ -v=/var/lib/lxc/:/var/lib/lxc/:shared \ -v=/var/lib/lxc/lxcfs/proc/diskstats:/proc/diskstats:rw,rslave \ -v=/var/lib/lxc/lxcfs/proc/meminfo:/proc/meminfo:rw,rslave \ -v=/var/lib/lxc/lxcfs/proc/stat:/proc/stat:rw,rslave \ -v=/var/lib/lxc/lxcfs/proc/swaps:/proc/swaps:rw,rslave \ -v=/var/lib/lxc/lxcfs/proc/uptime:/proc/uptime:rw,rslave \ -v=/var/lib/lxc/lxcfs/proc/loadavg:/proc/loadavg:rw,rslave \ --name ddd \ XXXXXXX 其中-v=/var/lib/lxc/:/var/lib/lxc/:shared \表示/var/lib/lxc/已经挂载

在重启lxcfs后，无论是在容器内mount还是用nsenter 都报错宿主机上： # nsenter --target 14591 --mount -- mount -B /var/lib/lxcfs/proc/meminfo /proc/meminfo mount: mount point /proc/meminfo does not exist

容器内： [root@master lxcfs]# mount -B "/var/lib/lxc/lxcfs/proc/cpuinfo" "/proc/cpuinfo" mount: mount point /proc/cpuinfo is not a directory

请问是否有设置不对的地方，还请指导一下，谢谢 @xigang

xigang commented 2 years ago

@fjibj 我采用如下方式启动容器 docker run -dt -m 5g \ -v=/sys/fs/cgroup:/sys/fs/cgroup:ro \ -v=/var/lib/lxc/:/var/lib/lxc/:shared \ -v=/var/lib/lxc/lxcfs/proc/diskstats:/proc/diskstats:rw,rslave \ -v=/var/lib/lxc/lxcfs/proc/meminfo:/proc/meminfo:rw,rslave \ -v=/var/lib/lxc/lxcfs/proc/stat:/proc/stat:rw,rslave \ -v=/var/lib/lxc/lxcfs/proc/swaps:/proc/swaps:rw,rslave \ -v=/var/lib/lxc/lxcfs/proc/uptime:/proc/uptime:rw,rslave \ -v=/var/lib/lxc/lxcfs/proc/loadavg:/proc/loadavg:rw,rslave \ --name ddd \ XXXXXXX 其中-v=/var/lib/lxc/:/var/lib/lxc/:shared \表示/var/lib/lxc/已经挂载

在重启lxcfs后，无论是在容器内mount还是用nsenter 都报错宿主机上： # nsenter --target 14591 --mount -- mount -B /var/lib/lxcfs/proc/meminfo /proc/meminfo mount: mount point /proc/meminfo does not exist

容器内： [root@master lxcfs]# mount -B "/var/lib/lxc/lxcfs/proc/cpuinfo" "/proc/cpuinfo" mount: mount point /proc/cpuinfo is not a directory

请问是否有设置不对的地方，还请指导一下，谢谢 @xigang

你直接使用文档中描述的方式，通过systemd启动lxcfs，尝试下？文档中描述的方式在生产环境是可行的

xigang / xigang.github.io

使用Lxcfs和kubernetes Admission Webhook实现对容器资源可视化隔离 | xigang's home #37