search

Quuxplusone / LLVMBugzillaTest

0 stars 0 forks source link

Race condition in compiler-rt cmake build #26053

Open Quuxplusone opened 8 years ago

Quuxplusone commented 8 years ago
Bugzilla Link PR26054
Status NEW
Importance P normal
Reported by H.J. Lu (hjl.tools@gmail.com)
Reported on 2016-01-06 18:26:52 -0800
Last modified on 2018-10-30 16:34:45 -0700
Version unspecified
Hardware PC Linux
CC adembo@gmail.com, diana.picus@linaro.org, llvm-bugs@lists.llvm.org, vitalybuka@google.com, vonosmas@gmail.com
Fixed by commit(s)
Attachments
Blocks
Blocked by
See also 
On machine with 24 cores,

cmake --build build-x86_64-linux-gnux32 -- -j 12

may lead to

/usr/local/bin/ld:/export/build/gnu/llvm-clang-x32-bootstrap-cmake/stage2/build-
x86_64-linux-gnux32/projects/compiler-rt/lib/asan/clang_rt.asan-dynamic-
x32.vers:1: syntax error in VERSION script
clang-3.8: error: linker command failed with exit code 1 (use -v to see
invocation)
projects/compiler-rt/lib/asan/CMakeFiles/clang_rt.asan-dynamic-
x32.dir/build.make:224: recipe for target
'lib/clang/3.8.0/lib/linux/libclang_rt.asan-x32.so' failed
gmake[5]: *** [lib/clang/3.8.0/lib/linux/libclang_rt.asan-x32.so] Error 1

libclang_rt.asan-x32.so was built while clang_rt.asan-dynamic-x32.vers
was being generated.  As the result, linker read the partially
written clang_rt.asan-dynamic-x32.vers.
Quuxplusone commented 8 years ago

+Evgeniy

I presume, you're building after r238367, and fix there doesn't work for you?

Quuxplusone commented 8 years ago 
(In reply to comment #1)
> +Evgeniy
>
> I presume, you're building after r238367, and fix there doesn't work for you?

No, r238367 doesn't fix it.
Quuxplusone commented 8 years ago 
I think I just ran into the same issue on an ARM machine:
/usr/bin/ld: cannot open linker script file
/home/linaro/devel/llvm/bisect/build/projects/compiler-
rt/lib/asan/clang_rt.asan-dynamic-arm.vers: No such file or directory
while trying to build lib/clang/4.0.0/lib/linux/libclang_rt.asan-arm.so. I was
using r279391.

This happened while doing essentially what the clang-cmake-thumbv7-a15-full-sh
selfhost bot is doing. I've done this build many times before (and the buildbot
it doing it all the time), so it's probably pretty rare.
Quuxplusone commented 6 years ago 
Is this still relevant?
How exactly to reproduce it?
Quuxplusone commented 5 years ago 
We have a job that builds LLVM 6.0.0 as part of a larger product and hits this
bug periodically. I've included a snippet of the failure at the end of this
comment.

I believe it's because of the "interesting" value of -j (and possibly --load-
average) that we pass into the build, which probably tickles a build race
somewhere.

> 2018-10-18 04:42:28.209582 /container.redhat7/build/kudu/thirdparty/build-
definitions.sh:339: make -j18 --load-average=36 install
> ...
> 2018-10-18 04:42:29.494939 Scanning dependencies of target clang_rt.asan-
dynamic-x86_64-version-list
> 2018-10-18 04:42:29.495482 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.497523 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.500315 [ 31%] Building C object projects/compiler-
rt/lib/builtins/CMakeFiles/clang_rt.builtins-x86_64.dir/mulxc3.c.o
> 2018-10-18 04:42:29.500370 [ 31%] Linking CXX static library
../../../../lib/clang/6.0.0/lib/linux/libclang_rt.ubsan_minimal-x86_64.a
> 2018-10-18 04:42:29.500382 [ 31%] Built target LLVMDebugInfoPDB
> 2018-10-18 04:42:29.500391 [ 31%] Building CXX object projects/compiler-
rt/lib/scudo/CMakeFiles/clang_rt.scudo-dynamic-x86_64.dir/scudo_tsd_shared.cpp.o
> 2018-10-18 04:42:29.500400 [ 31%] Building CXX object projects/compiler-
rt/lib/scudo/CMakeFiles/clang_rt.scudo-x86_64.dir/scudo_flags.cpp.o
> 2018-10-18 04:42:29.500408 [ 31%] Building C object projects/compiler-
rt/lib/builtins/CMakeFiles/clang_rt.builtins-x86_64.dir/powixf2.c.o
> 2018-10-18 04:42:29.500416 [ 31%] Built target clang_rt.safestack-x86_64
> 2018-10-18 04:42:29.500425 [ 31%] Built target profile
> 2018-10-18 04:42:29.500508 [ 31%] Building CXX object projects/compiler-
rt/lib/scudo/CMakeFiles/clang_rt.scudo-x86_64.dir/scudo_allocator.cpp.o
> 2018-10-18 04:42:29.501251 [ 31%] Linking CXX static library
../../../../../lib/liblldMachO.a
> 2018-10-18 04:42:29.501971 [ 31%] Generating version list for clang_rt.asan-
dynamic-x86_64
> 2018-10-18 04:42:29.504164 [ 32%] Building CXX object projects/compiler-
rt/lib/scudo/CMakeFiles/clang_rt.scudo-x86_64.dir/scudo_crc32.cpp.o
> 2018-10-18 04:42:29.504575 [ 32%] Building CXX object projects/compiler-
rt/lib/scudo/CMakeFiles/clang_rt.scudo-x86_64.dir/scudo_interceptors.cpp.o
> 2018-10-18 04:42:29.504717 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.505694 [ 32%] Building CXX object projects/compiler-
rt/lib/scudo/CMakeFiles/clang_rt.scudo-dynamic-x86_64.dir/scudo_utils.cpp.o
> 2018-10-18 04:42:29.508793 [ 32%] Building CXX object projects/compiler-
rt/lib/scudo/CMakeFiles/clang_rt.scudo-x86_64.dir/scudo_termination.cpp.o
> 2018-10-18 04:42:29.508955 [ 32%] Built target clang_rt.msan-x86_64-symbols
> 2018-10-18 04:42:29.510901 [ 32%] Building CXX object projects/compiler-
rt/lib/scudo/CMakeFiles/clang_rt.scudo-x86_64.dir/scudo_tsd_exclusive.cpp.o
> 2018-10-18 04:42:29.511668 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.511945 [ 32%] Linking C static library
../../../../lib/clang/6.0.0/lib/linux/libclang_rt.builtins-x86_64.a
> 2018-10-18 04:42:29.512224 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.514485 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.514775 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.514811 [ 32%] Building CXX object projects/compiler-
rt/lib/scudo/CMakeFiles/clang_rt.scudo-x86_64.dir/scudo_tsd_shared.cpp.o
> 2018-10-18 04:42:29.515331 [ 32%] Building CXX object projects/compiler-
rt/lib/scudo/CMakeFiles/clang_rt.scudo-dynamic-x86_64.dir/scudo_new_delete.cpp.o
> 2018-10-18 04:42:29.516563 [ 32%] Built target clang_rt.tsan-x86_64-symbols
> 2018-10-18 04:42:29.516849 [ 32%] Building CXX object projects/compiler-
rt/lib/scudo/CMakeFiles/clang_rt.scudo-x86_64.dir/scudo_utils.cpp.o
> 2018-10-18 04:42:29.517184 Scanning dependencies of target xray
> 2018-10-18 04:42:29.517710 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.517823 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.518770 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.519363 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.519399 [ 32%] Built target clang_rt.ubsan_minimal-x86_64
> 2018-10-18 04:42:29.519428 Scanning dependencies of target lsan
> 2018-10-18 04:42:29.520765 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.520798 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.521887 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.522575 Scanning dependencies of target
clang_rt.ubsan_standalone-x86_64-symbols
> 2018-10-18 04:42:29.523121 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.523350 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.524200 Scanning dependencies of target
RTAsan_dynamic_version_script_dummy.x86_64
> 2018-10-18 04:42:29.524243 [ 32%] Built target xray
> 2018-10-18 04:42:29.524263 Scanning dependencies of target
clang_rt.ubsan_standalone_cxx-x86_64-symbols
> 2018-10-18 04:42:29.524838 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.524950 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.525818 Scanning dependencies of target clang_rt.asan-
x86_64-symbols
> 2018-10-18 04:42:29.525968 [ 32%] Built target lsan
> 2018-10-18 04:42:29.526387 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.526865 [ 32%] Linking CXX shared library
../../../../lib/clang/6.0.0/lib/linux/libclang_rt.scudo-x86_64.so
> 2018-10-18 04:42:29.527593 Scanning dependencies of target clang_rt.asan_cxx-
x86_64-symbols
> 2018-10-18 04:42:29.527628 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.527953 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.528224 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.528346 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.528484 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.528712 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.528750 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.529085 [ 32%] Linking CXX static library
../../../../lib/clang/6.0.0/lib/linux/libclang_rt.scudo-x86_64.a
> 2018-10-18 04:42:29.529139 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.529232 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.529255 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.529961 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.530683 [ 32%] Built target clang_rt.ubsan_minimal-dynamic-
x86_64
> 2018-10-18 04:42:29.532471 [ 32%] Generating exported symbols for
clang_rt.ubsan_standalone-x86_64
> 2018-10-18 04:42:29.532505 [ 32%] Generating exported symbols for
clang_rt.ubsan_standalone_cxx-x86_64
> 2018-10-18 04:42:29.532517 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.532528 [ 32%] Generating exported symbols for
clang_rt.asan-x86_64
> 2018-10-18 04:42:29.532741 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.532783 [ 33%] Building CXX object projects/compiler-
rt/lib/asan/CMakeFiles/RTAsan_dynamic_version_script_dummy.x86_64.dir/dummy.cc.o
> 2018-10-18 04:42:29.533343 [ 33%] Built target LLVMSupport
> 2018-10-18 04:42:29.533998 Scanning dependencies of target dfsan
> 2018-10-18 04:42:29.534139 Scanning dependencies of target msan
> 2018-10-18 04:42:29.534592 [ 33%] Built target clang_rt.esan-x86_64-symbols
> 2018-10-18 04:42:29.534628 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.534663 [ 33%] Generating exported symbols for
clang_rt.asan_cxx-x86_64
> 2018-10-18 04:42:29.534788 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.534867 Scanning dependencies of target tsan
> 2018-10-18 04:42:29.535165 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.535565 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.536862 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.537563 [ 33%] Built target lldMachO
> 2018-10-18 04:42:29.538331 Scanning dependencies of target safestack
> 2018-10-18 04:42:29.539013 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.540023 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.540161 Scanning dependencies of target
clang_rt.ubsan_minimal-x86_64-symbols
> 2018-10-18 04:42:29.540265 [ 33%] Built target msan
> 2018-10-18 04:42:29.540442 [ 33%] Built target dfsan
> 2018-10-18 04:42:29.540692 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.540709 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.540864 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.541330 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.542679 [ 33%] Built target tsan
> 2018-10-18 04:42:29.545636 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.545700 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.545712 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.545721 [ 33%] Built target
RTAsan_dynamic_version_script_dummy.x86_64
> 2018-10-18 04:42:29.545730 [ 33%] Built target clang_rt.dyndd-dynamic-x86_64
> 2018-10-18 04:42:29.545857 Scanning dependencies of target clang-tblgen
> 2018-10-18 04:42:29.545934 [ 33%] Built target clang_rt.builtins-x86_64
> 2018-10-18 04:42:29.546435 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.546589 [ 33%] Built target safestack
> 2018-10-18 04:42:29.547551 [ 33%] Generating exported symbols for
clang_rt.ubsan_minimal-x86_64
> 2018-10-18 04:42:29.547585 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.547690 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.547986 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.548369 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.549576 Scanning dependencies of target llvm-config
> 2018-10-18 04:42:29.549616 Scanning dependencies of target llvm-tblgen
> 2018-10-18 04:42:29.549814 Scanning dependencies of target esan
> 2018-10-18 04:42:29.550182 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.550301 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.550513 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.552176 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.552303 Scanning dependencies of target clang_rt.asan-
dynamic-x86_64
> 2018-10-18 04:42:29.552398 Scanning dependencies of target dd
> 2018-10-18 04:42:29.552552 [ 33%] Linking CXX executable
../../../../bin/clang-tblgen
> 2018-10-18 04:42:29.552920 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.552958 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.553035 Scanning dependencies of target builtins
> 2018-10-18 04:42:29.553231 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.553648 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.555768 [ 33%] Built target esan
> 2018-10-18 04:42:29.556063 make[4]: Entering directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.556086 [ 33%] Linking CXX executable ../../bin/llvm-tblgen
> 2018-10-18 04:42:29.557472 [ 33%] Building CXX object tools/llvm-
config/CMakeFiles/llvm-config.dir/llvm-config.cpp.o
> 2018-10-18 04:42:29.558453 [ 33%] Built target dd
> 2018-10-18 04:42:29.559066 [ 33%] Built target builtins
> 2018-10-18 04:42:29.560467 [ 33%] Linking CXX shared library
../../../../lib/clang/6.0.0/lib/linux/libclang_rt.asan-x86_64.so
> 2018-10-18 04:42:29.571909 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.573561 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.573658 /container.redhat7/build/impala/toolchain/binutils-
2.26.1/bin/ld:/container.redhat7/build/kudu/thirdparty/build/llvm-
6.0.0/projects/compiler-rt/lib/asan/clang_rt.asan-dynamic-x86_64.vers:1: syntax
error in VERSION script
> 2018-10-18 04:42:29.573787 collect2: error: ld returned 1 exit status
> 2018-10-18 04:42:29.574264 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.575030 make[4]: ***
[lib/clang/6.0.0/lib/linux/libclang_rt.asan-x86_64.so] Error 1
> 2018-10-18 04:42:29.575086 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.575255 make[3]: *** [projects/compiler-
rt/lib/asan/CMakeFiles/clang_rt.asan-dynamic-x86_64.dir/all] Error 2
> 2018-10-18 04:42:29.575267 make[3]: *** Waiting for unfinished jobs....
> 2018-10-18 04:42:29.575798 [ 33%] Built target clang_rt.ubsan_standalone_cxx-
x86_64-symbols
> 2018-10-18 04:42:29.577618 [ 33%] Built target clang_rt.scudo-x86_64
> 2018-10-18 04:42:29.577672 [ 33%] Linking CXX executable ../../bin/llvm-config
> 2018-10-18 04:42:29.578077 [ 33%] Built target clang_rt.asan_cxx-x86_64-
symbols
> 2018-10-18 04:42:29.584646 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.588442 [ 33%] Built target clang_rt.ubsan_minimal-x86_64-
symbols
> 2018-10-18 04:42:29.592011 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.595997 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.596390 [ 33%] Built target clang_rt.ubsan_standalone-
x86_64-symbols
> 2018-10-18 04:42:29.600374 [ 33%] Built target clang_rt.asan-dynamic-x86_64-
version-list
> 2018-10-18 04:42:29.623119 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.623376 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.627202 [ 33%] Built target clang_rt.asan-x86_64-symbols
> 2018-10-18 04:42:29.627302 [ 33%] Built target clang_rt.scudo-dynamic-x86_64
> 2018-10-18 04:42:29.747747 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.751890 [ 33%] Built target llvm-config
> 2018-10-18 04:42:29.814763 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.818872 [ 33%] Built target clang-tblgen
> 2018-10-18 04:42:29.856173 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:29.860316 [ 33%] Built target llvm-tblgen
> 2018-10-18 04:42:33.161362 [ 33%] Linking CXX static library
../../libLLVMMCParser.a
> 2018-10-18 04:42:33.176762 make[4]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:33.180509 [ 33%] Built target LLVMMCParser
> 2018-10-18 04:42:33.181618 make[3]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:33.181925 make[2]: *** [all] Error 2
> 2018-10-18 04:42:33.182196 make[2]: Leaving directory
`/container.redhat7/build/kudu/thirdparty/build/llvm-6.0.0'
> 2018-10-18 04:42:33.182981 make[1]: *** [local-kudu-for-impala-internal]
Error 2
> 2018-10-18 04:42:33.183166 make[1]: Leaving directory `/container.redhat7'
> 2018-10-18 04:42:33.183608 make: *** [local-kudu-for-impala] Error 2
Quuxplusone commented 5 years ago 
I have managed to reproduce this fairly reliably with LLVM 6.0.0.

Steps for reproduction:
1. Extract an LLVM 6.0.0 tarball, with corresponding compiler-rt in
projects/compiler-rt.
2. Create a build directory outside the extracted tarball.
3. Patch projects/compiler-rt/lib/sanitizer_common/scripts/gen_dynamic_list.py
with something like the following:

--- /tmp/gen_dynamic_list.py    2018-10-30 16:31:21.086781701 -0700
+++ ../../src/llvm-6.0.0.src/projects/compiler-
rt/lib/sanitizer_common/scripts/gen_dynamic_list.py 2018-10-30
15:31:45.314698264 -0700
@@ -118,6 +118,8 @@
       result.append(line.rstrip())
   # Print the resulting list in the format recognized by ld.
   print('{')
+  import time
+  time.sleep(5)
   if args.version_list:
     print('global:')
   result.sort()

4. Change the working directory to the build directory created in step 2.
5. Run "cmake -DCMAKE_BUILD_TYPE=Release -DLLVM_TARGETS_TO_BUILD=X86
/path/to/extracted/llvm/tarball". I am using cmake 3.9.0 and a gcc 4.9.2
toolchain, as well as a local installation of Python 2.7.14 (the system-
provided Python is too old).
6. Run "NUM_RUNS=0; while (cd projects/compiler-rt && make clean && make -j12);
do NUM_RUNS=$(($NUM_RUNS + 1)); done; echo $NUM_RUNS runs passed before
failure". With a hot ccache, I can reproduce the failure within 1-2 runs. This
is a CentOS 6.6 machine with 48 cores that is otherwise idle.

I haven't managed to repro the issue when I run with -j48, -j24, or -j8. I also
haven't managed to repro it using ninja instead of make. I think -j12 is the
key to the bug, or at least one of several possible keys.