Long build line can hit MAX_ARG_STRLEN on Linux

[AndreasKempfNEC]

Dear Ninja team,

(edit: damn, I should have used the search function. It seems that this is a known issue: https://github.com/ninja-build/ninja/pull/520 https://github.com/ninja-build/ninja/issues/53 So should this be fixed in Meson, then?)

while trying to port a large codebase (> 10000 files) to the Meson build system on Linux, I noticed that really long build lines can fail without providing an error message explaining the problem. This can easily be replicated by, e.g., the following (useless) Ninja build file with files main.c and test.c:

cc = gcc 

rule build
  command = $cc $in -o $out

build prog: build main.c test.c test.c test.c ... [replicate thousands of times]

$ ninja
... test.c [replicate thousands of times] ... -o prog
ninja: build stopped: subcommand failed.

More specifically, it seems to fail with line lengths longer than around 128 kiB. Having done some digging, I suspect that Ninja hits the MAX_ARG_STRLEN limit of the Linux kernel [1, 2, 3],

Number of arguments and maximum length of one argument At least on Linux 2.6, there's also a limit on the maximum number of arguments in argv[]. On Linux 2.6.14 the function do_execve() in fs/exec.c tests if the number exceeds

PAGE_SIZEMAX_ARG_PAGES-sizeof(void ) / sizeof(void *)

On a 32-bit Linux, this is ARGMAX/4-1 (32767). This becomes relevant if the average length of arguments is smaller than 4.

Since Linux 2.6.23, this function tests if the number exceeds MAX_ARG_STRINGS in <linux/binfmts.h> (2^32-1 = 4294967296-1).

And as additional limit since 2.6.23, one argument must not be longer than MAX_ARG_STRLEN (131072). This might become relevant if you generate a long call like "sh -c 'automatically generated with many arguments'". (pointed out by Xan Lopez and Ralf Wildenhues)

Without really knowing too much about the Ninja source, I would suspect the following lines to cause this,

const char* spawned_args[] = { "/bin/sh", "-c", command.c_str(), NULL };
  if (posix_spawn(&pid_, "/bin/sh", &action, &attr,
                  const_cast<char**>(spawned_args), environ) != 0)
Fatal("posix_spawn: %s", strerror(errno));

https://github.com/ninja-build/ninja/blob/9e71431e6f8323be8ced8997409cfe7a389c6583/src/subprocess-posix.cc#L103

It seems that the build line is passed to /bin/sh as a single command line argument. As explained above, this might hit a limit at some point.

Is this analysis correct? If so, do I have to work around this limitation somehow or could this be changed in a future Ninja version? If I understand correctly, the argument will have to be split, or the options will have to be passed via STDIN for really long commands to work.

[1] https://www.in-ulm.de/~mascheck/various/argmax/ [2] http://lxr.free-electrons.com/source/include/uapi/linux/binfmts.h#L14 [3] short test:

$ /bin/echo "$(printf "%*s" 131071 ".")">/dev/null
$ /bin/echo "$(printf "%*s" 131072 ".")">/dev/null
-bash: /bin/echo: Argument list too long

[nico]

Generators can use rspfile, rspfile_contents (https://ninja-build.org/manual.html#ref_rule) to make things go with very long command lines.

Looks like our error message could be better here though, so I'm leaving this open for that.

[dankegel]

See https://github.com/mesonbuild/meson/pull/7245

[Germi1981]

Hello,

I have some troubles regarding this error

posix_spawn: Argument list too long 07:12:24 ninja failed with: exit status 1

failed to build some targets (4 seconds)

Any solution for Ubuntu 20.04?

Thanks in advance

[BehroozAmoozad]

So, did anyone figure this out yet?

[0/8662] Compiling Vala source ../src/<REDACTED>
ninja: fatal: posix_spawn: Argument list too long

[VTVishwanath]

I am also facing same kind of issue. Any solution?