landley
commented
9 months ago The most recent commit to that file was 4 and 1/2 years ago, and we've had 10 releases since then.
The failure is that your libc's strndup() returned NULL. I have two questions about this, both of which are "how?"
The first "how?" is that on an allyesconfig build "toybox help | wc -c" says 154k total help text. For comparison, when I run my devuan x86-64's "sleep" command /proc/$PID/smaps_rollup says 1592 kB referenced. If running this binary exhausts memory, you couldn't have compiled it. (Did you dynamically set some crazy-low ulimit that's an order of magnitude lower than what the compiler is allowed to consume? Some sort of container environment bug, maybe?)
The second "how?" is that running out of physical memory has only ever returned NULL from libc allocation functions on NOMMU Linux. With an MMU, Linux libc suballocates from the heap and extends it by allocating more virtual address space, then the physical memory is dynamically allocated later by the page fault handler during a write access to the COW mapping of the zero page (or the modern eqvivalent that's more SMP friendly and avoids the reference count wrapping issue big iron hit, but conceptually the same). The OOM killer picks a process to SIGKILL at this time if it can't satisfy the allocation, usually the one that asked for more memory but there's a bucket of terrible heuristics that gets stirred from time to time to make it bigger.
Recognition of "we actually don't have enough memory" happpens long after the allocation of virtual space has returned, unless you twiddle some "make the system extremely unstable" knobs ("disable overcommit") under /proc which basically ask programs to speculatively fail when they otherwise would have succeeded. (I.E. it never makes MORE stuff work, it only ever makes LESS stuff work. It changes the Linux programming API.)
Toybox proper always checks those return values (xmalloc() and friends) because it supports running on NOMMU systems that can arbitrarily return NULL due to memory fragmentation and similar, and because commands exiting with failure happens all the time, ala "echo > /dev/full" and so on. But we don't support building on nommu systems, which is why I didn't bother to check for a NULL return from allocations in build tools processing known inputs: if it does fail, all I can do is exit() because the system is completely horked. I could change the line to exit with a more graceful error message, but your build would still break.
It's theoretically possible that the failing call to strndup() has some other reason for returning NULL? Maybe it got dodgy arguments (a negative length maybe?) and I'm curious what its inputs were when it returned NULL. It's also possible that the heap was corrupted at some point before then and the malloc failed chasing pointers, except A) glibc dies with an assert() when that happens (rather than allocation functions randomly returning NULL), B) you ran it under valgrind which should catch any out of bounds write even to elsewhere in the heap if it's not cleanly within an allocation boundary.
When scripts/make.sh calls mkflags it snapshots its input with tee, producing generated/flags.raw for later analysis, because flag processing is tricksy and occasionally non-obvious and has gone wrong a few times over the years (albeit when adding new commands, not when building a release version on someone else's system). But I never bothered doing that with config2help.c because in the 9 years since I replaced the python build script with a C implementation it's literally never come up.
Still, if you wanted to add a call to tee into the pipeline on scripts/make.sh line 266 and send me the input going to config2help, I can try to reproduce your issue here?
thesamesam
I originally reported this downstream in Gentoo at https://bugs.gentoo.org/914589.
If I jam HOSTCC="gcc -ggdb3", I get this when running
generated/unstripped/config2help Config.in .config:Possibly slightly more useful from valgrind:
From asan/ubsan (probably the most helpful, as you get the naughtiness immediately rather than when it tries to read later on):