RAM and swap revert to the full amount available at the host

[AhmadFazliIsmail]

Hi,

I am running CentOS 7 kernel 3.10.0-693.2.2.el7.x86_64 and manually compile LXCFS (have just download the current release today) using ./configure --bindir=/bin --sbindir=/sbin --with-init-script=systemd --libdir=/usr/lib --with-pamdir=none

I installed lxc-1.0.10-2.el7.x86_64 from EPEL repo using yum

The installation was successful. I created a container and run it using the config file below;

[root@lxc01 ~]# cat /var/lib/lxc/test/config 
# Template used to create this container: /usr/share/lxc/templates/lxc-centos
# Parameters passed to the template:
# For additional config options, please look at lxc.container.conf(5)
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = virbr0
lxc.network.hwaddr = fe:5d:ae:bc:43:19
lxc.rootfs = /var/lib/lxc/test/rootfs

# Include common configuration
lxc.include = /usr/share/lxc/config/centos.common.conf

lxc.arch = x86_64
lxc.utsname = test

lxc.autodev = 1

# When using LXC with apparmor, uncomment the next line to run unconfined:
#lxc.aa_profile = unconfined

# example simple networking setup, uncomment to enable
#lxc.network.type = veth
#lxc.network.flags = up
#lxc.network.link = lxcbr0
#lxc.network.name = eth0
# Additional example for veth network type
#    static MAC address,
#lxc.network.hwaddr = 00:16:3e:77:52:20
#    persistent veth device name on host side
#        Note: This may potentially collide with other containers of same name!
#lxc.network.veth.pair = v-test-e0

lxc.cgroup.cpuset.cpus                 = 1
lxc.cgroup.memory.limit_in_bytes       = 2G
lxc.cgroup.memory.memsw.limit_in_bytes = 4G

lxc.mount.auto = cgroup:mixed
lxc.autodev = 1
lxc.kmsg = 0
lxc.include = /usr/local/share/lxc/config/common.conf.d/00-lxcfs.conf
#lxc.include = /usr/share/lxc/config/common.conf.d/00-lxcfs.conf

It is showing the correct uptime, cpuinfo, and memory when it started as shown below;

[root@test ~]# free -mt
              total        used        free      shared  buff/cache   available
Mem:           2048          17        2022          16           8        2022
Swap:          2048           0        2048
Total:         4096          17        4070
[root@test ~]# uptime
 18:00:58 up 0 min,  1 user,  load average: 0.11, 0.04, 0.05

I let it run for about 2 hours, check their status and found;

[root@test etc]# free -mt  
              total        used        free      shared  buff/cache   available
Mem:          15870           2       15834          16          33       15834
Swap:          8187           0        8187
Total:        24058           2       24022
[root@test etc]# uptime 
 20:17:39 up  2:17,  1 user,  load average: 0.00, 0.01, 0.05

It is showing the RAM and Swap at the host and not the container. Appreciate any advice on this matter.

Thanks.

[hallyn]

Hi,

can you test with the kernel-lt or kernel-ml package?

If that works, then it's a sign that https://bugs.centos.org/view.php?id=13265&nbn=9 is not yet fixed in the default Centos kernel.

[AhmadFazliIsmail]

Thanks @hallyn for your prompt response. I am still performing test using my current kernel but that issue seems has disappear! /var/log/messages and dmesg for that container does not show any weird logs. Besides that was the 1st run after installing LXCFS, now is the 2nd run. Will continuously test and monitor. Not yet try kernel-lt or kernel-ml, may try on kernel-lt later.

Thanks again for your advice.

[AhmadFazliIsmail]

After some checking, found this in the host /var/log/messages;

Oct 18 22:18:14 lxc01 python: SELinux is preventing /usr/bin/find from using the dac_read_search capability.#012#012***** Plugin dac_override (91.4 confidence) suggests **********************#012#012If you want to help identify if domain needs this access or you have a file with the wrong permissions on your system#012Then turn on full auditing to get path information about the offending file and generate the error again.#012Do#012#012Turn on full auditing#012# auditctl -w /etc/shadow -p w#012Try to recreate AVC. Then execute#012# ausearch -m avc -ts recent#012If you see PATH record check ownership/permissions on file, and fix it,#012otherwise report as a bugzilla.#012#012***** Plugin catchall (9.59 confidence) suggests **************************#012#012If you believe that find should have the dac_read_search capability by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'find' --raw | audit2allow -M my-find#012# semodule -i my-find.pp#012

Then after doing that sealert and permit that blocking, this issue occurred again.

Now the host is using kernel-ml Linux lxc01.local.domain 4.13.7-1.el7.elrepo.x86_64 #1 SMP Sat Oct 14 11:46:13 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux. I reboot, start that container, test the uptime, cpuinfo and meminfo, and this issue not yet happen. Look like this issue is related to the kernel version and SELinux. However will continuously test the container and report the result here within next week.

Thank you once again @hallyn for your advise to update the kernel.

[AhmadFazliIsmail]

As previously promised, I come again with a report during the test that was performed last week.

I have to integrate LXC to Libvirt-LXC because;

1. That same issue still existed even using kernel-ml or kernel-lt
2. VNIC of the containers sometimes need to be removed manually after they powered off.
3. The IP addresses of the containers are still in used (I can ping them) even after they were powered off.
4. Libvirt-LXC is easier to use since there is GUI software / virt-manager instead of command-line. Besides I can manage them along with other KVMs.

I am suspecting that issue is related to LXC and not LXCFS. As mentioned above I am using lxc-1.0.10-2.el7.x86_64 from EPEL repo. There are no problem for LXCFS to run in Libvirt-LXC so I think that switching to Libvirt-LXC with LXCFS is the best option for me.

Many thank you for the great LXCFS. I can make my container running like KVM.