search

lxc / lxcfs

FUSE filesystem for LXC
https://linuxcontainers.org/lxcfs
Other
1.05k stars 250 forks source link

LXCFS slows down Wine in container a lot #659

Closed eebssk1 closed 1 month ago

eebssk1 commented 2 months ago

The template below is mostly useful for bug reports and support questions. Feel free to remove anything which doesn't apply to you and add more information where it makes sense.

Required information

sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=49410536k,nr_inodes=12352634,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=9886504k,mode=755,inode64 0 0 /dev/mapper/sys-root / ext4 rw,relatime,nombcache,errors=remount-ro,commit=7,min_batch_time=250,max_batch_time=11000,stripe=128 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 efivarfs /sys/firmware/efi/efivars efivarfs rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=55427 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,nosuid,nodev,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 /dev/mapper/data-vol /mnt/data ext4 rw,relatime,errors=remount-ro,commit=8,min_batch_time=300,max_batch_time=10000,stripe=256 0 0 /dev/nvme7n1p2 /boot ext2 rw,relatime 0 0 /dev/nvme7n1p1 /boot/efi vfat rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 lxcfs /var/lib/incus-lxcfs fuse.lxcfs rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other 0 0 tmpfs /var/lib/incus/shmounts tmpfs rw,relatime,size=100k,mode=711,inode64 0 0 tmpfs /var/lib/incus/guestapi tmpfs rw,relatime,size=100k,mode=755,inode64 0 0 /dev/mapper/data-vol /var/lib/incus/storage-pools/alternate ext4 rw,relatime,errors=remount-ro,commit=8,min_batch_time=300,max_batch_time=10000,stripe=256 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=9886500k,nr_inodes=2471625,mode=700,uid=1000,gid=1000,inode64 0 0

root 1506 17.7 0.0 751304 1536 ? Ssl Sep09 237:40 /opt/incus/bin/lxcfs /var/lib/incus-lxcfs


# Issue description

exe runs in wine takes a lot time to startup, make it impratical to use in lxc.

A brief description of what failed or what could be improved.

If you have LXCFS crashing, please, collect a crash dump.

# Steps to reproduce

just start one or more exe through wine.

lxcfs takes 60%~150% cpu usage when exe is starting.
and exe takes a lot secs to startup

# Information to attach
It looks like most time are spend at reading cpu info in loop.see attached picture
![图片](https://github.com/user-attachments/assets/c90ca676-b5d3-46be-8087-3b7415015c82)

 - [ ] any relevant kernel output (`dmesg`)
 - [ ] LXCFS daemon output / logs
 - [ ] LXCFS configuration (Which options were used to start a LXCFS daemon? `ps aux | grep lxcfs`)
 - [ ] in case of crash, a core dump (please, read [how to collect a core dump](https://github.com/lxc/lxcfs?tab=readme-ov-file#core-dump))
eebssk1 commented 2 months ago

Just a thought: It may be better to cache contents that won't change normally,so we can return them directly instead of reading everytime to avoid overhead.(Maybe create a indepent thread to watch for possible changes that seldom happen,like hotplug event in physical machine).

eebssk1 commented 2 months ago

I added

[Service]
CPUSchedulingPolicy=rr
CPUSchedulingPriority=2

via a systemd drop in file and the speed of wine improved a lot.

Now every executable in wine takes 1-2 secs to start. Which is normal in wine.

stgraber commented 1 month ago

LXCFS doesn't really know (or wants to know) what file may or may not change and may or may not return varying values based on the reader's namespace or other process attribute.

Storing cached data on a per-process basis is also impractical as it would then require LXCFS to keep track of processes still running and would offer a pretty easy way to DoS the host system.

As annoying as high CPU usage of LXCFS may be, it just slows things down. It doesn't allow for data leakage (as caching might) nor causing a full host DoS (as per-PID caching might).

LXCFS never reads files on its own, it always happens as a result of something being read through its FUSE filesystem. So in this case, wine is what you should strace to track down what LXCFS file it's abusing. You can then either try to get wine upstream to be maybe a bit less aggressive on reads to those files (which are typically single-threaded in-kernel, so not constantly reading them will help outside containers too) or you can chose to unmount LXCFS for that particular file in the container to restore host level performance (but at the cost of inaccurate resource information).

eebssk1 commented 1 month ago

Thanks for the explanation! Anyway I'm no longer running any heavy tool like msys2 under wine anymore, just too painful to use. I'll give instances.lxcfs.per_instance=true a try if needed to seprate different workloads.

Since this issue is very much the same as #655 , feel free to close or merge!