memory leaks found by ASAN (-fsanitize=address)

[matthiaskrgr]

Basically, compile add "-fsanitize=address,undefined -g3 -fno-omit-frame-pointer" to your CFLAGS and "-fsanitize=address,undefined" to your LDFLAGS, build and run ninja.

To do this, I edited the configure.py script since I didn't know any better.

These leaks were reported while bootstrapping ninja:

==25691==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 1400 byte(s) in 25 object(s) allocated from:
    #0 0x4ea86b in operator new(unsigned long) /home/matthias/LLVM/LLVM_3_9/stage_2/llvm/projects/compiler-rt/lib/asan/asan_new_delete.cc:78:35
    #1 0x5b9507 in BuildLog::RecordCommand(Edge*, int, int, int) /home/matthias/vcs/github/ninja/src/build_log.cc:159:19
    #2 0x55d4d9 in Builder::FinishCommand(CommandRunner::Result*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) /home/matthias/vcs/github/ninja/src/build.cc:851:29
    #3 0x556bf3 in Builder::Build(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) /home/matthias/vcs/github/ninja/src/build.cc:694:12
    #4 0x4f6775 in (anonymous namespace)::NinjaMain::RunBuild(int, char**) /home/matthias/vcs/github/ninja/src/ninja.cc:990:16
    #5 0x4ef2db in (anonymous namespace)::real_main(int, char**) /home/matthias/vcs/github/ninja/src/ninja.cc:1182:24
    #6 0x4ecabd in main /home/matthias/vcs/github/ninja/src/ninja.cc:1211:10
    #7 0x7f5929ed5400 in __libc_start_main /usr/src/debug/glibc-2.24-33-ge9e69e4/csu/../csu/libc-start.c:289

Direct leak of 352 byte(s) in 22 object(s) allocated from:
    #0 0x4ea86b in operator new(unsigned long) /home/matthias/LLVM/LLVM_3_9/stage_2/llvm/projects/compiler-rt/lib/asan/asan_new_delete.cc:78:35
    #1 0x5f44d3 in DepsLog::RecordDeps(Node*, int, int, Node**) /home/matthias/vcs/github/ninja/src/deps_log.cc:150:16
    #2 0x5f2f44 in DepsLog::RecordDeps(Node*, int, std::vector<Node*, std::allocator<Node*> > const&) /home/matthias/vcs/github/ninja/src/deps_log.cc:82:10
    #3 0x55e0d2 in Builder::FinishCommand(CommandRunner::Result*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) /home/matthias/vcs/github/ninja/src/build.cc:864:28
    #4 0x556bf3 in Builder::Build(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) /home/matthias/vcs/github/ninja/src/build.cc:694:12
    #5 0x4f6775 in (anonymous namespace)::NinjaMain::RunBuild(int, char**) /home/matthias/vcs/github/ninja/src/ninja.cc:990:16
    #6 0x4ef2db in (anonymous namespace)::real_main(int, char**) /home/matthias/vcs/github/ninja/src/ninja.cc:1182:24
    #7 0x4ecabd in main /home/matthias/vcs/github/ninja/src/ninja.cc:1211:10
    #8 0x7f5929ed5400 in __libc_start_main /usr/src/debug/glibc-2.24-33-ge9e69e4/csu/../csu/libc-start.c:289

Direct leak of 296 byte(s) in 1 object(s) allocated from:
    #0 0x4ea86b in operator new(unsigned long) /home/matthias/LLVM/LLVM_3_9/stage_2/llvm/projects/compiler-rt/lib/asan/asan_new_delete.cc:78:35
    #1 0x5528e1 in Builder::Builder(State*, BuildConfig const&, BuildLog*, DepsLog*, DiskInterface*) /home/matthias/vcs/github/ninja/src/build.cc:572:13
    #2 0x4f62e3 in (anonymous namespace)::NinjaMain::RunBuild(int, char**) /home/matthias/vcs/github/ninja/src/ninja.cc:969:11
    #3 0x4ef2db in (anonymous namespace)::real_main(int, char**) /home/matthias/vcs/github/ninja/src/ninja.cc:1182:24
    #4 0x4ecabd in main /home/matthias/vcs/github/ninja/src/ninja.cc:1211:10
    #5 0x7f5929ed5400 in __libc_start_main /usr/src/debug/glibc-2.24-33-ge9e69e4/csu/../csu/libc-start.c:289

Direct leak of 296 byte(s) in 1 object(s) allocated from:
    #0 0x4ea86b in operator new(unsigned long) /home/matthias/LLVM/LLVM_3_9/stage_2/llvm/projects/compiler-rt/lib/asan/asan_new_delete.cc:78:35
    #1 0x5528e1 in Builder::Builder(State*, BuildConfig const&, BuildLog*, DepsLog*, DiskInterface*) /home/matthias/vcs/github/ninja/src/build.cc:572:13
    #2 0x4f56d8 in (anonymous namespace)::NinjaMain::RebuildManifest(char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) /home/matthias/vcs/github/ninja/src/ninja.cc:243:11
    #3 0x4eef7b in (anonymous namespace)::real_main(int, char**) /home/matthias/vcs/github/ninja/src/ninja.cc:1170:15
    #4 0x4ecabd in main /home/matthias/vcs/github/ninja/src/ninja.cc:1211:10
    #5 0x7f5929ed5400 in __libc_start_main /usr/src/debug/glibc-2.24-33-ge9e69e4/csu/../csu/libc-start.c:289

The "ninja_test" binary also reported a lot of leaks, I am not gonna flood the ticket with these

[sgraham]

I believe the leaks are intentional as ninja is not (currently) intended for use in a daemon or library mode, so simple process termination is faster/simpler than releasing memory.

[matthiaskrgr]

For the record, the two biggest leaks in the sample above add up to quite a bit when building larger projects. This is an empty (rebuild with everything already being compiled) run on llvm/clang:

Direct leak of 154168 byte(s) in 2753 object(s) allocated from:
    #0 0x4ea86b in operator new(unsigned long) /home/matthias/LLVM/LLVM_3_9/stage_2/llvm/projects/compiler-rt/lib/asan/asan_new_delete.cc:78:35
    #1 0x5bc1a9 in BuildLog::Load(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) /home/matthias/vcs/github/ninja/src/build_log.cc:309:15
    #2 0x4f31b1 in (anonymous namespace)::NinjaMain::OpenBuildLog(bool) /home/matthias/vcs/github/ninja/src/ninja.cc:875:19
    #3 0x4ee872 in (anonymous namespace)::real_main(int, char**) /home/matthias/vcs/github/ninja/src/ninja.cc:1163:16
    #4 0x4ecabd in main /home/matthias/vcs/github/ninja/src/ninja.cc:1211:10
    #5 0x7fbee1baf400 in __libc_start_main /usr/src/debug/glibc-2.24-33-ge9e69e4/csu/../csu/libc-start.c:289

Direct leak of 38224 byte(s) in 2389 object(s) allocated from:
    #0 0x4ea86b in operator new(unsigned long) /home/matthias/LLVM/LLVM_3_9/stage_2/llvm/projects/compiler-rt/lib/asan/asan_new_delete.cc:78:35
    #1 0x5f7283 in DepsLog::Load(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, State*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) /home/matthias/vcs/github/ninja/src/deps_log.cc:225:20
    #2 0x4f44e9 in (anonymous namespace)::NinjaMain::OpenDepsLog(bool) /home/matthias/vcs/github/ninja/src/ninja.cc:910:18
    #3 0x4ee8e0 in (anonymous namespace)::real_main(int, char**) /home/matthias/vcs/github/ninja/src/ninja.cc:1163:41
    #4 0x4ecabd in main /home/matthias/vcs/github/ninja/src/ninja.cc:1211:10
    #5 0x7fbee1baf400 in __libc_start_main /usr/src/debug/glibc-2.24-33-ge9e69e4/csu/../csu/libc-start.c:289

[nico]

What sgraham said. The two leaks you mention allocate data that's needed during the build.

[evmar]

If ASAN can find definitely-leaked memory (allocated memory that has no references) as distinct from memory that is still referenced at shutdown that would be valuable. I seem to recall valgrind makes this distinction.

On Feb 23, 2017 9:52 AM, "Nico Weber" notifications@github.com wrote:

What sgraham said. The two leaks you mention allocate data that's needed during the build.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ninja-build/ninja/issues/1247#issuecomment-282067932, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAPB9j4DxBnV3cuktI0aNSxIqQIr7maks5rfcdAgaJpZM4MJZoT .

[matthiaskrgr]

ASAN does report "direct leaks" and "indirect leaks" is this what you mean? I just did not add the indirect leaks to the report because there were way too much to have them inlined.

[jonesmz]

@matthiaskrgr Is this still reproducible with Ninja 1.10.0?

[eugenis]

If ASAN can find definitely-leaked memory (allocated memory that has no references)

This is exactly what ASan finds. Memory that is still referenced at shutdown (i.e. transitively reachable from a number of root regions through anything that looks like a pointer) in not a leak. It is OK to exit without deallocating everything.