ninja-build / ninja

a small build system with a focus on speed
https://ninja-build.org/
Apache License 2.0
10.96k stars 1.58k forks source link

"ninja: build stopped: subcommand failed." - which subcommand ? No clue in log. #2378

Open JVD66 opened 7 months ago

JVD66 commented 7 months ago

Please, improve ninja to say WHAT subcommand failed ; simply exiting with "ninja: build stopped: subcommand failed." is useless - it doesn't tell people WHAT failed.

My log file, for a GCC build, shows NO error messages from GCC, so I'm stuck.

I have to run the whole build with 'strace -s8192 -f -e trace=execve' and tediously analyse this log to find which sub-processes / commands have failed.

This is unacceptable - I'd recommend converting projects from ninja use to plain GNU Make use or CMake use because of this issue (and many others).

digit-google commented 7 months ago

You may be misunderstood. Ninja already prints the subcommand which failed before this message (as well as the combined stdout + stderr of that command). You can verify that by writing the following to a build.ninja file the invoking Ninja in the same directory:

rule foo
  command = echo "to_stdout"; echo >&2 "to_stderr"; exit 1

build out: foo

You will get:

[1/1] echo "to_stdout"; echo >&2 "to_stderr"; exit 1
FAILED: out 
echo "to_stdout"; echo >&2 "to_stderr"; exit 1
to_stdout
to_stderr
ninja: build stopped: subcommand failed.

However, the failed command + its outputs are sent to stderr, while the "ninja: build stopped: subcommand failed" message is sent to stdout.

Did you redirect stderr to a different file before invoking Ninja? That would explain what you're are seeing.

johnkrah commented 5 months ago

@digit-google the example you've given is clearly true but I think it is too simple to attempt to simulate the user feedback. Problem that @JVD66 is describing and I can reproduce is something like the following:

  1. have a build with many targets and use a high number of concurrent tasks, e.g. -j 300
  2. one of those targets fails (naturally or synthetically)
  3. ninja pipes stdout and stderr to console/log for any and all tasks that have started 3.1. and consider the case where there is non-trivial amount of info printed for many targets
  4. the failing target is usually not the last target to complete 4.1. almost perforce since the failing target will signal to ninja to stop and other subcommands will either stop after signaled or stop when naturally complete
  5. this results in the information necessary to debug the problem is far away from the end of console/log

it's also true that with knowledge that ninja will output the string FAILED: out one can search regular expression ^FAILED: (?<target>.+)$ or whatever other kind of string search to find that line and the subommand's actual output should be near it as well.

requested feature improvement is for ninja to output a summary statement identifying which subcommand failed. using the same example you supplied, I'd add that the desired output is:

ninja: build stopped: subcommand failed: out.

or simpler alternative that is a constant log message and leaves searching to the user but gives a gentle nudge in the right direction:

ninja: build stopped: subcommand failed. Look for "FAILED: <build target name>" above.
zougloub commented 4 months ago

If not re-printing failing commands, please remind us to look for FAILED when a failure occurs. This is really a PITA when compiling with a large number of jobs.

summerfind commented 3 months ago

Hi, I just build hello_world and got error: FAILED: hello_world.elf ---- collect2.exe: error: ld returned 1 exit status --- ninja: build stopped: subcommand failed. how to fix? Thanks

rustyhowell-c4 commented 1 month ago

Because ninja output can be very large, it is difficult to find the error message in an active terminal window. It would be great if ninja would write the entire log to a log file all the time. Yes I know stdout and stderr can be redirected, but so often it's a pain to rerun ninja just to capture the error.

Another option would be to write just the stdout of the failing thread to a log file. That would be helpful as well.

flexibeast commented 3 weeks ago

requested feature improvement is for ninja to output a summary statement identifying which subcommand failed

To which i'd add: "together with the error message and/or code associated with the failure." It seems make-work to me to force people to speculate as to what the reason(s) for the failure might be if that information has been provided by the subcommand, and makes it more difficult to assist people who are experiencing compilation failures.