Ninja hides the exit status of failing subprocesses

[jonesmz]

In build.cc, Ninja checks if the subprocess succeeded or failed

  // Print the command that is spewing before printing its output.
  if (!success) {
    string outputs;
    for (vector<Node*>::const_iterator o = edge->outputs_.begin();
         o != edge->outputs_.end(); ++o)
      outputs += (*o)->path() + " ";

    printer_.PrintOnNewLine("FAILED: " + outputs + "\n");
    printer_.PrintOnNewLine(edge->EvaluateCommand() + "\n");
  }

But by this point in the build, Ninja has already discarded the actual exit value from the subprocess.

There are a variety of tools that don't print anything to the console when they fail to run, relying on the user to instead check the exit status to see what the reason for failure was.

This is especially problematic for third party tools where the source code is not available.

It's, of course, possible to get the exit status printed to the console using a wrapper script, but as discussed at extreme length, wrapper scripts are desired by basically no one.

Ninja should print the exit status of failing subprocesses.

[jimis]

I don't see any "FAILED:" message, but instead ninja: build stopped: subcommand failed

Is it the same issue?

@schultetwin1 Would https://github.com/ninja-build/ninja/pull/1805 address this situation too, and reveal the exit code of the failed subcommand?

[schultetwin1]

@jimis if you are not seeing any "FAILED:" message this seems like a different issue. #1805 only adds to the "FAILED:" string. #1805 has no changes that would cause that "FAILED:" string to appear when it currently does not.

[jimis]

Thanks @schultetwin1. Indeed it seems the "subcommand failed" message is coming from https://github.com/ninja-build/ninja/blob/54959b0f2c4950d97d94c03810b3b5185be0d69e/src/build.cc#L883 which is not touched in your patch.

The effect is the same however: an exit code that is lost, while it could have been printed.

[jonatino]

@jonesmz @jimis @schultetwin1 have you guys figured this out?

My make file:

#!/bin/bash
# Build script
export CC=/usr/bin/clang
export CXX=/usr/bin/clang++
mkdir -p build
cd build
cmake .. -G "Ninja" -DCMAKE_BUILD_TYPE=Release
ninja
cd ..

Error in github actions (it still thinks the build is successful):

Run ./make
-- The C compiler identification is AppleClang 12.0.0.12000032
-- The CXX compiler identification is AppleClang 12.0.0.12000032
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Applications/Xcode_12.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode_12.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Build spdlog: 1.8.2
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Build type: Release
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/runner/work/pvphero-launcher/pvphero-launcher/build
[1/20] Copying OS X Content PvPHero.app/Contents/Resources/icon.icns
[2/20] Building CXX object deps/spdlog/CMakeFiles/spdlog.dir/src/file_sinks.cpp.o
[3/20] Building CXX object deps/spdlog/CMakeFiles/spdlog.dir/src/stdout_sinks.cpp.o
[4/20] Building CXX object deps/spdlog/CMakeFiles/spdlog.dir/src/color_sinks.cpp.o
[5/20] Building CXX object deps/spdlog/CMakeFiles/spdlog.dir/src/spdlog.cpp.o
[6/20] Building CXX object CMakeFiles/PvPHero.dir/src/updating/Downloader.cpp.o
[7/20] Building CXX object CMakeFiles/PvPHero.dir/src/updating/client/ClientUpdater.cpp.o
[8/20] Building CXX object CMakeFiles/PvPHero.dir/src/PvPHero.cpp.o
[9/20] Building CXX object CMakeFiles/PvPHero.dir/src/external/sago/platform_folders.cpp.o
[10/20] Building CXX object CMakeFiles/PvPHero.dir/src/filesystem/InMemoryFileSystem.cpp.o
[11/20] Building CXX object CMakeFiles/PvPHero.dir/src/updating/Utils.cpp.o
../src/updating/Utils.cpp:61:39: warning: object backing the pointer will be destroyed at the end of the full-expression [-Wdangling-gsl]
        const char *command_line[] = {getJavaPath().c_str(), "-m", "pvphero/com.pvphero.PvPHero",
                                      ^~~~~~~~~~~~~
1 warning generated.
[12/20] Building C object CMakeFiles/PvPHero.dir/src/external/zip/zip.c.o
[13/20] Building CXX object CMakeFiles/PvPHero.dir/src/Assets.cpp.o
../src/Assets.cpp:2:9: warning: #pragma once in main file [-Wpragma-once-outside-header]
#pragma once
        ^
1 warning generated.
[14/20] Building CXX object deps/spdlog/CMakeFiles/spdlog.dir/src/async.cpp.o
[15/20] Building CXX object deps/spdlog/CMakeFiles/spdlog.dir/src/cfg.cpp.o
[16/20] Building CXX object CMakeFiles/PvPHero.dir/src/updating/launcher/LauncherUpdater.cpp.o
[17/20] Building CXX object deps/spdlog/CMakeFiles/spdlog.dir/src/fmt.cpp.o
[18/20] Linking CXX static library deps/spdlog/libspdlog.a
[19/20] Building CXX object CMakeFiles/PvPHero.dir/src/main.cpp.o
[20/20] Linking CXX executable PvPHero.app/Contents/MacOS/PvPHero
FAILED: PvPHero.app/Contents/MacOS/PvPHero 
: && /Applications/Xcode_12.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -O3 -DNDEBUG -isysroot /Applications/Xcode_12.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.0.sdk -mmacosx-version-min=10.15 -Wl,-search_paths_first -Wl,-headerpad_max_install_names /SUBSYSTEM:CONSOLE -s CMakeFiles/PvPHero.dir/src/PvPHero.cpp.o CMakeFiles/PvPHero.dir/src/updating/Utils.cpp.o CMakeFiles/PvPHero.dir/src/updating/Downloader.cpp.o CMakeFiles/PvPHero.dir/src/updating/client/ClientUpdater.cpp.o CMakeFiles/PvPHero.dir/src/updating/launcher/LauncherUpdater.cpp.o CMakeFiles/PvPHero.dir/src/filesystem/InMemoryFileSystem.cpp.o CMakeFiles/PvPHero.dir/src/external/sago/platform_folders.cpp.o CMakeFiles/PvPHero.dir/src/external/zip/zip.c.o CMakeFiles/PvPHero.dir/src/Assets.cpp.o CMakeFiles/PvPHero.dir/src/main.cpp.o -o PvPHero.app/Contents/MacOS/PvPHero -L/Users/runner/work/pvphero-launcher/pvphero-launcher/build/../deps/AppCore/lib   -L/Users/runner/work/pvphero-launcher/pvphero-launcher/build/../deps/Ultralight/lib   -L/Users/runner/work/pvphero-launcher/pvphero-launcher/build/../deps/UltralightCore/lib   -L/Users/runner/work/pvphero-launcher/pvphero-launcher/build/../deps/WebCore/lib   -L/Users/runner/work/pvphero-launcher/pvphero-launcher/build/../deps/WebKitLibraries/lib -Wl,-rpath,@executable_path/  -lWebCore  -lUltralightCore  -lUltralight  -lAppCore  -lblend2d  -lWTF  -lJavaScriptCore  -lPAL  -lsqlite3  ../deps//WebKitLibraries/lib/libbrotlidec.a  ../deps//WebKitLibraries/lib/libbrotlienc.a  ../deps//WebKitLibraries/lib/libbrotlicommon.a  ../deps//WebKitLibraries/lib/libcrypto.a  ../deps//WebKitLibraries/lib/libcurl.a  ../deps//WebKitLibraries/lib/libfreetype.a  ../deps//WebKitLibraries/lib/libharfbuzz.a  ../deps//WebKitLibraries/lib/libharfbuzz-icu.a  ../deps//WebKitLibraries/lib/libicudata.a  ../deps//WebKitLibraries/lib/libicui18n.a  ../deps//WebKitLibraries/lib/libicuuc.a  ../deps//WebKitLibraries/lib/libjpeg.a  ../deps//WebKitLibraries/lib/libnghttp2_static.a  ../deps//WebKitLibraries/lib/libpng.a  ../deps//WebKitLibraries/lib/libpng16.a  ../deps//WebKitLibraries/lib/libssl.a  ../deps//WebKitLibraries/lib/libtls.a  ../deps//WebKitLibraries/lib/libxml2.a  ../deps//WebKitLibraries/lib/libxslt.a  ../deps//WebKitLibraries/lib/libz.a  -lpthread  -lbmalloc  -ldl  -lbz2  -framework  Accelerate  -framework  AppKit  -framework  SystemConfiguration  -framework  Security && :
clang: error: no such file or directory: '/SUBSYSTEM:CONSOLE'
ninja: build stopped: subcommand failed.

[jimis]

@Jonatino you have a clear indication of what's wrong in your case:

clang: error: no such file or directory: '/SUBSYSTEM:CONSOLE'

@jonesmz @jimis @schultetwin1 have you guys figured this out?

Please file separate issues for irrelevant problems.

[jonatino]

@jimis thats obvious. The problem is that github CI does not fail because that exit code is hidden because its a ninja subproces... which is exactly what this issue is reporting.

[jimis]

@Jonatino ninja exits with its own error code in case of failure. It is your bash script that hides that. Try adding set -e at the beginning of your script and it will exit on the first error. It has nothing to do with this ticket or in general with ninja.

[mgorny]

I've been hit hard with this when ccache started SIGBUS-ing on me and I had no way of figuring out why all the compilation jobs are suddenly failing with no output.

[cassandracomar]

@jonatino ninja exits with its own error code in case of failure. It is your bash script that hides that. Try adding set -e at the beginning of your script and it will exit on the first error. It has nothing to do with this ticket or in general with ninja.

I have set -e set, check for the exit status of ninja and return it, and yet my script continues on as if ninja's exit code were zero. I'm pretty sure this ticket is valid.