Closed gibmat closed 1 year ago
Hi Mathias!
Thanks for your report, I'll take a look and fix it if you don't want to send a fix by yourself.
I wasn't sure what the preferred way to fix this would be (add a similar get_procfile_size_with_personality()
method?), so I don't have any patch to submit.
Introduction
Earlier this year it was reported that on certain Debian armel/armhf builders,
/proc/cpuinfo
was empty (bug report). Eventually this was tracked down to occurring in lxc containers running armel/armhf builds on arm64 hosts. (lxc version 5.0.2 and lxcfs version 5.0.3.)Today, I finally had time to sit down, spin up VMs in QEMU and identify the root cause: the fix for #553 contains an edge case bug. When there are more than ~13 CPUs available to an arm64 host, any 32bit arm containers will fail to provide a populated
/proc/cpuinfo
due to exhausting the cache inproc_cpuinfo_read()
.Problem
Code in the arm64 kernel (link) checks the process' personality when called, and if it's 32bit, introduces another line to each CPU reported in
/proc/cpuinfo
(for examplemodel name : ARMv8 Processor rev 1 (v8l)
). That's about 40 bytes additional output per CPU that doesn't appear if called from a process with a 64bit personality. (The reported feature flags will also change, but at least in my QEMU setup the overall length of that line was about the same for 32bit and 64bit invocations.)Currently, within
proc_cpuinfo_read()
the variablecache_size
is set from the value ofd->buflen
, which is initialized insrc/proc_fuse.c
asinfo->buflen = get_procfile_size(path) + BUF_RESERVE_SIZE
.get_procfile_size()
simply gobbles up the specified file and reports its total size, but does so without accounting for any personality differences.Therefore, when lxcfs sets up the expected size of the buffer for
/proc/cpuinfo
on an arm64 host, it unconditionally sets the buffer to expect the output of a 64bit system. When an armel/armhf container comes along, the result that is actually read from the host's/proc/cpuinfo
is ~40 bytes longer per CPU which eats intoBUF_RESERVE_SIZE
(512 bytes), causingproc_cpuinfo_read()
to fail if there are more than 512/~40 = ~13 CPUs on the host system.Reproducing
QEMU is limited to 8 CPUs for an arm VM, so I simply shrunk the value of
BUF_RESERVE_SIZE
from 512 to 128. With that change, I can successfully read/proc/cpuinfo
from a 32bit container with up to 3 CPUs on the arm64 host, but bumping up to 4 CPUs breaks in the same way as reported in the original Debian bug.Fix
get_procfile_size()
needs to be made aware of personalities as well, similar to the logic inproc_read_with_personality()
.(edit to fix typo: sysfile -> procfile... whoops!)