Quuxplusone / LLVMBugzillaTest

0 stars 0 forks source link

non-deterministic stage2/stage3 file comparison failures in 3 stage bootstrap on x86_64 darwin and x86_64 linux #31314

Closed Quuxplusone closed 7 years ago

Quuxplusone commented 7 years ago
Bugzilla Link PR32342
Status RESOLVED FIXED
Importance P release blocker
Reported by Jack Howarth (howarth.mailing.lists@gmail.com)
Reported on 2017-03-18 15:20:36 -0700
Last modified on 2017-04-16 19:14:40 -0700
Version trunk
Hardware Macintosh MacOS X
CC dcoughlin@apple.com, ditaliano@apple.com, fang@csl.cornell.edu, ghoflehner@apple.com, iains-llvm@btconnect.com, joker.eph@gmail.com, llvm-bugs@lists.llvm.org
Fixed by commit(s)
Attachments DependenceAnalysis.cpp.o.stage2 (145888 bytes, application/octet-stream)
DependenceAnalysis.cpp.o.stage3 (145872 bytes, application/octet-stream)
asan_activation.cc.o.stage2 (51076 bytes, application/octet-stream)
asan_activation.cc.o.stage3 (51076 bytes, application/octet-stream)
ELFDumper.cpp.o.dis.diff.bz2 (381615 bytes, application/octet-stream)
SourceCoverageViewHTML.ii.bz2 (201545 bytes, application/octet-stream)
ClangTidyDiagnosticConsumer.ii.bz2 (385701 bytes, application/octet-stream)
ELFDumper.ii.bz2 (247172 bytes, application/octet-stream)
ELFDumper.s.diff.linux (4863 bytes, text/plain)
SourceCoverageViewHTML.s.diff.linux (784 bytes, text/plain)
ClangTidyDiagnosticConsumer.s.diff.linux (1052 bytes, text/plain)
FastISelEmitter.ii.bz2 (195659 bytes, application/octet-stream)
FastISelEmitter.s.non-determinism (93497 bytes, text/plain)
Blocks
Blocked by
See also
The commit of...

------------------------------------------------------------------------
r296837 | ddcc | 2017-03-02 18:05:45 -0500 (Thu, 02 Mar 2017) | 9 lines

Reland 3: [analyzer] NFC: Update test infrastructure to support multiple
constraint managers

Summary: Replace calls to %clang/%clang_cc1 with %clang_analyze_cc1 when
invoking static analyzer, and perform runtime substitution to select the
appropriate constraint manager, per D28952.

Reviewers: xazax.hun, NoQ, zaks.anna, dcoughlin

Subscribers: mgorny, rgov, mikhail.ramalho, a.sidorin, cfe-commits

Differential Revision: https://reviews.llvm.org/D30373

triggers a number of new stage2/stage3 file comparison failures in a 3 stage
bootstrap of llvm/clang/clang-tool-extras/compiler-rt/openmp/polly/libc++ on
x86_64-apple-darwain16.

UNRESOLVED DIFFS (stage2 vs. stage3):
  lib/Analysis/CMakeFiles/LLVMAnalysis.dir/DependenceAnalysis.cpp.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_activation.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_allocator.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_debugging.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_descriptions.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_errors.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_fake_stack.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_flags.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_globals.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_globals_win.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_interceptors.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_linux.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_mac.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_malloc_linux.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_malloc_mac.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_malloc_win.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_memory_profile.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_new_delete.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_poisoning.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_posix.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_report.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_rtl.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_stack.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_stats.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_suppressions.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_thread.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_win.cc.o
  projects/compiler-rt/lib/interception/CMakeFiles/RTInterception.osx.dir/interception_linux.cc.o
  projects/compiler-rt/lib/interception/CMakeFiles/RTInterception.osx.dir/interception_mac.cc.o
  projects/compiler-rt/lib/interception/CMakeFiles/RTInterception.osx.dir/interception_type_test.cc.o
  projects/compiler-rt/lib/interception/CMakeFiles/RTInterception.osx.dir/interception_win.cc.o
  projects/compiler-rt/lib/lsan/CMakeFiles/RTLSanCommon.osx.dir/lsan_common.cc.o
  projects/compiler-rt/lib/lsan/CMakeFiles/RTLSanCommon.osx.dir/lsan_common_linux.cc.o
  projects/compiler-rt/lib/lsan/CMakeFiles/RTLSanCommon.osx.dir/lsan_common_mac.cc.o
  projects/compiler-rt/lib/safestack/CMakeFiles/clang_rt.safestack_osx.dir/safestack.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_allocator.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_common.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_deadlock_detector1.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_deadlock_detector2.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_flag_parser.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_flags.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_libc.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_libignore.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_linux.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_linux_s390.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_mac.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_persistent_allocator.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_platform_limits_linux.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_platform_limits_posix.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_posix.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_printf.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_procmaps_common.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_procmaps_freebsd.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_procmaps_linux.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_procmaps_mac.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_stackdepot.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_stacktrace.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_stacktrace_printer.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_stoptheworld_mac.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_suppressions.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_symbolizer.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_symbolizer_libbacktrace.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_symbolizer_mac.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_symbolizer_win.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_termination.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_thread_registry.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_tls_get_addr.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_win.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sancov_flags.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_common_libcdep.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_coverage_libcdep.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_coverage_libcdep_new.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_coverage_mapping_libcdep.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_coverage_win_sections.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_linux_libcdep.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_posix_libcdep.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_stacktrace_libcdep.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_stoptheworld_linux_libcdep.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_symbolizer_libcdep.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_symbolizer_posix_libcdep.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_unwind_linux_libcdep.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoLibc.osx.dir/sanitizer_common_nolibc.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_allocator.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_common.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_deadlock_detector1.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_deadlock_detector2.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_flag_parser.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_flags.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_libc.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_libignore.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_linux.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_linux_s390.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_mac.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_persistent_allocator.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_platform_limits_linux.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_platform_limits_posix.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_posix.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_printf.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_procmaps_common.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_procmaps_freebsd.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_procmaps_linux.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_procmaps_mac.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_stackdepot.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_stacktrace.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_stacktrace_printer.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_stoptheworld_mac.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_suppressions.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_symbolizer.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_symbolizer_libbacktrace.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_symbolizer_mac.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_symbolizer_win.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_thread_registry.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_tls_get_addr.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_win.cc.o
  projects/compiler-rt/lib/stats/CMakeFiles/clang_rt.stats_client_osx.dir/stats_client.cc.o
  projects/compiler-rt/lib/stats/CMakeFiles/clang_rt.stats_osx_dynamic.dir/stats.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_clock.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_debugging.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_external.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_fd.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_flags.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_ignoreset.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_interceptors.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_interceptors_mac.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_interface.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_interface_ann.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_interface_atomic.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_interface_java.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_libdispatch_mac.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_malloc_mac.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_md5.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_mman.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_mutex.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_mutexset.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_new_delete.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_platform_mac.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_platform_posix.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_preinit.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_report.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_rtl.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_rtl_amd64.S.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_rtl_mutex.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_rtl_proc.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_rtl_report.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_rtl_thread.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_stack_trace.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_stat.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_suppressions.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_symbolize.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_sync.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_clock.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_debugging.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_external.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_fd.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_flags.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_ignoreset.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_interceptors.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_interceptors_mac.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_interface.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_interface_ann.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_interface_atomic.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_interface_java.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_libdispatch_mac.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_malloc_mac.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_md5.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_mman.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_mutex.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_mutexset.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_new_delete.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_platform_mac.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_platform_posix.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_preinit.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_report.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_rtl.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_rtl_amd64.S.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_rtl_mutex.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_rtl_proc.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_rtl_report.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_rtl_thread.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_stack_trace.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_stat.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_suppressions.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_symbolize.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_sync.cc.o
  projects/compiler-rt/lib/ubsan/CMakeFiles/RTUbsan.osx.dir/ubsan_diag.cc.o
  projects/compiler-rt/lib/ubsan/CMakeFiles/RTUbsan.osx.dir/ubsan_flags.cc.o
  projects/compiler-rt/lib/ubsan/CMakeFiles/RTUbsan.osx.dir/ubsan_handlers.cc.o
  projects/compiler-rt/lib/ubsan/CMakeFiles/RTUbsan.osx.dir/ubsan_handlers_cxx.cc.o
  projects/compiler-rt/lib/ubsan/CMakeFiles/RTUbsan.osx.dir/ubsan_init.cc.o
  projects/compiler-rt/lib/ubsan/CMakeFiles/RTUbsan.osx.dir/ubsan_type_hash.cc.o
  projects/compiler-rt/lib/ubsan/CMakeFiles/RTUbsan.osx.dir/ubsan_type_hash_itanium.cc.o
  projects/compiler-rt/lib/ubsan/CMakeFiles/RTUbsan.osx.dir/ubsan_type_hash_win.cc.o
  projects/compiler-rt/lib/ubsan/CMakeFiles/RTUbsan.osx.dir/ubsan_value.cc.o
  projects/compiler-rt/lib/ubsan/CMakeFiles/RTUbsan_standalone.osx.dir/ubsan_init_standalone.cc.o
  projects/openmp/runtime/src/kmp_i18n_default.inc
  projects/openmp/runtime/src/kmp_i18n_id.inc
  tools/llvm-config/BuildVariables.inc
  tools/llvm-config/CMakeFiles/llvm-config.dir/llvm-config.cpp.o
Quuxplusone commented 7 years ago

Attached DependenceAnalysis.cpp.o.stage3 (145872 bytes, application/octet-stream): object file for stage3/lib/Analysis/CMakeFiles/LLVMAnalysis.dir/DependenceAnalysis.cpp.o on x86_64-apple-darwin16 at r296837

Quuxplusone commented 7 years ago

Attached DependenceAnalysis.cpp.o.stage2 (145888 bytes, application/octet-stream): object file for stage2/lib/Analysis/CMakeFiles/LLVMAnalysis.dir/DependenceAnalysis.cpp.o on x86_64-apple-darwin16 at r296837

Quuxplusone commented 7 years ago

Attached asan_activation.cc.o.stage2 (51076 bytes, application/octet-stream): object file for stage2/projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_activation.cc.o on x86_64-apple-darwin16

Quuxplusone commented 7 years ago

Attached asan_activation.cc.o.stage3 (51076 bytes, application/octet-stream): object file for stage3/projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_activation.cc.o on x86_64-apple-darwin16

Quuxplusone commented 7 years ago
Jack,

I think it is very unlikely that r296837 introduced a difference between
stage2/stage3. That commit changed only clang static analyzer tests and the
lit.cfg for clang tests. Are you sure it was that commit?

Here is the list of files changed:

test/Analysis/CFContainers-invalid.c
test/Analysis/CFContainers.mm
test/Analysis/CFDateGC.m
test/Analysis/CFNumber.c
test/Analysis/CFRetainRelease_NSAssertionHandler.m
test/Analysis/CGColorSpace.c
test/Analysis/CheckNSError.m
test/Analysis/DeallocMissingRelease.m
test/Analysis/DeallocUseAfterFreeErrors.m
test/Analysis/DynamicTypePropagation.m
test/Analysis/Malloc+MismatchedDeallocator+NewDelete.cpp
test/Analysis/Malloc+MismatchedDeallocator_intersections.cpp
test/Analysis/Malloc+NewDelete_intersections.cpp
test/Analysis/MemRegion.cpp
test/Analysis/MismatchedDeallocator-checker-test.mm
test/Analysis/MismatchedDeallocator-path-notes.cpp
test/Analysis/MissingDealloc.m
test/Analysis/NSContainers.m
test/Analysis/NSPanel.m
test/Analysis/NSString.m
test/Analysis/NSWindow.m
test/Analysis/NewDelete+MismatchedDeallocator_intersections.cpp
test/Analysis/NewDelete-checker-test.cpp
test/Analysis/NewDelete-custom.cpp
test/Analysis/NewDelete-intersections.mm
test/Analysis/NewDelete-path-notes.cpp
test/Analysis/NewDelete-variadic.cpp
test/Analysis/NewDeleteLeaks-PR18394.cpp
test/Analysis/NewDeleteLeaks-PR19102.cpp
test/Analysis/NoReturn.m
test/Analysis/OSAtomic_mac.cpp
test/Analysis/ObjCProperties.m
test/Analysis/ObjCPropertiesSyntaxChecks.m
test/Analysis/ObjCRetSigs.m
test/Analysis/PR12905.c
test/Analysis/PR24184.cpp
test/Analysis/PR2599.m
test/Analysis/PR2978.m
test/Analysis/PR3991.m
test/Analysis/PR7218.c
test/Analysis/additive-folding-range-constraints.c
test/Analysis/additive-folding.cpp
test/Analysis/analyzeOneFunction.m
test/Analysis/analyzer-checker-config.c
test/Analysis/analyzer-config.c
test/Analysis/analyzer-config.cpp
test/Analysis/analyzer-display-progress.cpp
test/Analysis/analyzer-display-progress.m
test/Analysis/analyzer-enabled-checkers.c
test/Analysis/analyzer-stats.c
test/Analysis/array-struct-region.c
test/Analysis/array-struct-region.cpp
test/Analysis/array-struct.c
test/Analysis/atomics.c
test/Analysis/auto-obj-dtors-cfg-output.cpp
test/Analysis/base-init.cpp
test/Analysis/bitwise-ops.c
test/Analysis/block-in-critical-section.cpp
test/Analysis/blocks-no-inline.c
test/Analysis/blocks.m
test/Analysis/blocks.mm
test/Analysis/bool-assignment.c
test/Analysis/bstring.c
test/Analysis/bstring.cpp
test/Analysis/bug_hash_test.cpp
test/Analysis/bug_hash_test.m
test/Analysis/builtin-functions.cpp
test/Analysis/call-invalidation.cpp
test/Analysis/cast-to-struct.cpp
test/Analysis/castexpr-callback.c
test/Analysis/casts.c
test/Analysis/casts.cpp
test/Analysis/casts.m
test/Analysis/cfg.cpp
test/Analysis/cfref_PR2519.c
test/Analysis/cfref_rdar6080742.c
test/Analysis/check-deserialization.cpp
test/Analysis/checker-plugins.c
test/Analysis/chroot.c
test/Analysis/comparison-implicit-casts.cpp
test/Analysis/complex-init-list.cpp
test/Analysis/complex.c
test/Analysis/concrete-address.c
test/Analysis/conditional-operator.cpp
test/Analysis/conditional-path-notes.c
test/Analysis/const-method-call.cpp
test/Analysis/constant-folding.c
test/Analysis/conversion.c
test/Analysis/copypaste/asm.cpp
test/Analysis/copypaste/attributes.cpp
test/Analysis/copypaste/blocks.cpp
test/Analysis/copypaste/call.cpp
test/Analysis/copypaste/catch.cpp
test/Analysis/copypaste/delete.cpp
test/Analysis/copypaste/dependent-exist.cpp
test/Analysis/copypaste/expr-types.cpp
test/Analysis/copypaste/fold.cpp
test/Analysis/copypaste/function-try-block.cpp
test/Analysis/copypaste/functions.cpp
test/Analysis/copypaste/generic.c
test/Analysis/copypaste/labels.cpp
test/Analysis/copypaste/lambda.cpp
test/Analysis/copypaste/macro-complexity.cpp
test/Analysis/copypaste/macros.cpp
test/Analysis/copypaste/objc-methods.m
test/Analysis/copypaste/plist-diagnostics-notes-as-events.cpp
test/Analysis/copypaste/plist-diagnostics.cpp
test/Analysis/copypaste/sub-sequences.cpp
test/Analysis/copypaste/suspicious-clones.cpp
test/Analysis/copypaste/text-diagnostics.cpp
test/Analysis/coverage.c
test/Analysis/crash-trace.c
test/Analysis/cstring-syntax-cxx.cpp
test/Analysis/cstring-syntax.c
test/Analysis/ctor.mm
test/Analysis/cxx-crashes.cpp
test/Analysis/cxx-for-range.cpp
test/Analysis/cxx-method-names.cpp
test/Analysis/cxx11-crashes.cpp
test/Analysis/dead-stores.c
test/Analysis/dead-stores.cpp
test/Analysis/dead-stores.m
test/Analysis/debug-CallGraph.c
test/Analysis/default-analyze.m
test/Analysis/default-diagnostic-visitors.c
test/Analysis/delayed-template-parsing-crash.cpp
test/Analysis/delegates.m
test/Analysis/derived-to-base.cpp
test/Analysis/designated-initializer.c
test/Analysis/diagnostics/deref-track-symbolic-region.c
test/Analysis/diagnostics/deref-track-symbolic-region.cpp
test/Analysis/diagnostics/diag-cross-file-boundaries.c
test/Analysis/diagnostics/explicit-suppression.cpp
test/Analysis/diagnostics/false-positive-suppression.c
test/Analysis/diagnostics/implicit-cxx-std-suppression.cpp
test/Analysis/diagnostics/macros.cpp
test/Analysis/diagnostics/macros.m
test/Analysis/diagnostics/no-prune-paths.c
test/Analysis/diagnostics/plist-diagnostics-include-check.cpp
test/Analysis/diagnostics/report-issues-within-main-file.cpp
test/Analysis/diagnostics/shortest-path-suppression.c
test/Analysis/diagnostics/text-diagnostics.c
test/Analysis/diagnostics/undef-value-caller.c
test/Analysis/diagnostics/undef-value-param.c
test/Analysis/diagnostics/undef-value-param.m
test/Analysis/disable-all-checks.c
test/Analysis/dispatch-once.m
test/Analysis/div-zero.cpp
test/Analysis/division-by-zero.c
test/Analysis/domtest.c
test/Analysis/dtor-cxx11.cpp
test/Analysis/dtor.cpp
test/Analysis/dtors-in-dtor-cfg-output.cpp
test/Analysis/dynamic-cast.cpp
test/Analysis/dynamic_type_check.m
test/Analysis/edges-new.mm
test/Analysis/elementtype.c
test/Analysis/engine/replay-without-inlining.c
test/Analysis/enum.cpp
test/Analysis/exceptions.mm
test/Analysis/exercise-ps.c
test/Analysis/explain-svals.cpp
test/Analysis/explain-svals.m
test/Analysis/expr-inspection.c
test/Analysis/fields.c
test/Analysis/free.c
test/Analysis/func.c
test/Analysis/generics.m
test/Analysis/global-region-invalidation.c
test/Analysis/global_region_invalidation.mm
test/Analysis/gtest.cpp
test/Analysis/html-diags-multifile.c
test/Analysis/html-diags.c
test/Analysis/identical-expressions.cpp
test/Analysis/index-type.c
test/Analysis/initializer.cpp
test/Analysis/initializers-cfg-output.cpp
test/Analysis/inline-not-supported.c
test/Analysis/inline-plist.c
test/Analysis/inline-unique-reports.c
test/Analysis/inline.c
test/Analysis/inline.cpp
test/Analysis/inline2.c
test/Analysis/inline3.c
test/Analysis/inline4.c
test/Analysis/inlining/DynDispatchBifurcate.m
test/Analysis/inlining/InlineObjCClassMethod.m
test/Analysis/inlining/InlineObjCInstanceMethod.m
test/Analysis/inlining/ObjCDynTypePopagation.m
test/Analysis/inlining/ObjCImproperDynamictallyDetectableCast.m
test/Analysis/inlining/RetainCountExamples.m
test/Analysis/inlining/analysis-order.c
test/Analysis/inlining/assume-super-init-does-not-return-nil.m
test/Analysis/inlining/containers.cpp
test/Analysis/inlining/dyn-dispatch-bifurcate.cpp
test/Analysis/inlining/eager-reclamation-path-notes.c
test/Analysis/inlining/eager-reclamation-path-notes.cpp
test/Analysis/inlining/false-positive-suppression.c
test/Analysis/inlining/false-positive-suppression.cpp
test/Analysis/inlining/false-positive-suppression.m
test/Analysis/inlining/inline-defensive-checks.c
test/Analysis/inlining/inline-defensive-checks.cpp
test/Analysis/inlining/inline-defensive-checks.m
test/Analysis/inlining/path-notes.c
test/Analysis/inlining/path-notes.cpp
test/Analysis/inlining/path-notes.m
test/Analysis/inlining/retain-count-self-init.m
test/Analysis/inlining/stl.cpp
test/Analysis/inlining/test-always-inline-size-option.c
test/Analysis/inlining/test_objc_inlining_option.m
test/Analysis/iterator-past-end.cpp
test/Analysis/ivars.m
test/Analysis/keychainAPI-diagnostic-visitor.m
test/Analysis/keychainAPI.m
test/Analysis/kmalloc-linux.c
test/Analysis/lambda-notes.cpp
test/Analysis/lambdas-generalized-capture.cpp
test/Analysis/lambdas.cpp
test/Analysis/lambdas.mm
test/Analysis/lifetime-extension.cpp
test/Analysis/lit.local.cfg
test/Analysis/live-variables.cpp
test/Analysis/live-variables.m
test/Analysis/localization-aggressive.m
test/Analysis/localization.m
test/Analysis/logical-ops.c
test/Analysis/loop-widening.c
test/Analysis/lvalue.cpp
test/Analysis/malloc-annotations.c
test/Analysis/malloc-custom.c
test/Analysis/malloc-interprocedural.c
test/Analysis/malloc-overflow.c
test/Analysis/malloc-overflow.cpp
test/Analysis/malloc-overflow2.c
test/Analysis/malloc-plist.c
test/Analysis/malloc-protoype.c
test/Analysis/malloc-sizeof.c
test/Analysis/malloc-sizeof.cpp
test/Analysis/malloc-three-arg.c
test/Analysis/malloc.c
test/Analysis/malloc.cpp
test/Analysis/malloc.m
test/Analysis/malloc.mm
test/Analysis/max-nodes-suppress-on-sink.c
test/Analysis/member-expr.cpp
test/Analysis/method-call-intra-p.cpp
test/Analysis/method-call-path-notes.cpp
test/Analysis/method-call.cpp
test/Analysis/misc-ps-64.m
test/Analysis/misc-ps-arm.m
test/Analysis/misc-ps-cxx0x.cpp
test/Analysis/misc-ps-eager-assume.m
test/Analysis/misc-ps-ranges.m
test/Analysis/misc-ps-region-store-i386.m
test/Analysis/misc-ps-region-store-x86_64.m
test/Analysis/misc-ps-region-store.cpp
test/Analysis/misc-ps-region-store.m
test/Analysis/misc-ps-region-store.mm
test/Analysis/misc-ps.c
test/Analysis/misc-ps.m
test/Analysis/model-file.cpp
test/Analysis/mpichecker.cpp
test/Analysis/mpicheckernotes.cpp
test/Analysis/new-with-exceptions.cpp
test/Analysis/new.cpp
test/Analysis/nil-receiver-undefined-larger-than-voidptr-ret-region.m
test/Analysis/nil-receiver-undefined-larger-than-voidptr-ret.m
test/Analysis/no-exit-cfg.c
test/Analysis/no-outofbounds.c
test/Analysis/no-unreachable-dtors.cpp
test/Analysis/non-diagnosable-assumptions.c
test/Analysis/nonnull.m
test/Analysis/null-deref-path-notes.m
test/Analysis/null-deref-ps-region.c
test/Analysis/null-deref-ps.c
test/Analysis/nullability-no-arc.mm
test/Analysis/nullability.c
test/Analysis/nullability.mm
test/Analysis/nullability_nullonly.mm
test/Analysis/nullptr.cpp
test/Analysis/number-object-conversion.c
test/Analysis/number-object-conversion.cpp
test/Analysis/number-object-conversion.m
test/Analysis/objc-arc.m
test/Analysis/objc-bool.m
test/Analysis/objc-boxing.m
test/Analysis/objc-for.m
test/Analysis/objc-message.m
test/Analysis/objc-method-coverage.m
test/Analysis/objc-properties.m
test/Analysis/objc-radar17039661.m
test/Analysis/objc-string.mm
test/Analysis/objc-subscript.m
test/Analysis/objc/direct-ivar-assignment-in-annotated-functions.m
test/Analysis/objc_invalidation.m
test/Analysis/operator-calls.cpp
test/Analysis/out-of-bounds-new.cpp
test/Analysis/out-of-bounds.c
test/Analysis/outofbound-notwork.c
test/Analysis/outofbound.c
test/Analysis/override-werror.c
test/Analysis/padding_c.c
test/Analysis/padding_cpp.cpp
test/Analysis/padding_message.cpp
test/Analysis/plist-html-macros.c
test/Analysis/plist-macros.cpp
test/Analysis/plist-output-alternate.m
test/Analysis/plist-output.m
test/Analysis/pointer-to-member.cpp
test/Analysis/pr22954.c
test/Analysis/pr4209.m
test/Analysis/pr_2542_rdar_6793404.m
test/Analysis/pr_4164.c
test/Analysis/properties.m
test/Analysis/properties.mm
test/Analysis/pthreadlock.c
test/Analysis/ptr-arith.c
test/Analysis/ptr-arith.cpp
test/Analysis/qt_malloc.cpp
test/Analysis/range_casts.c
test/Analysis/rdar-6442306-1.m
test/Analysis/rdar-6540084.m
test/Analysis/rdar-6541136-region.c
test/Analysis/rdar-6562655.m
test/Analysis/rdar-6600344-nil-receiver-undefined-struct-ret.m
test/Analysis/rdar-7168531.m
test/Analysis/redefined_system.c
test/Analysis/refcnt_naming.m
test/Analysis/reference.cpp
test/Analysis/reference.mm
test/Analysis/region-1.m
test/Analysis/region-store.c
test/Analysis/region-store.cpp
test/Analysis/reinterpret-cast.cpp
test/Analysis/retain-release-arc.m
test/Analysis/retain-release-cache-out.m
test/Analysis/retain-release-cf-audited.m
test/Analysis/retain-release-gc-only.m
test/Analysis/retain-release-inline.m
test/Analysis/retain-release-path-notes-gc.m
test/Analysis/retain-release-path-notes.m
test/Analysis/retain-release-region-store.m
test/Analysis/retain-release.m
test/Analysis/retain-release.mm
test/Analysis/return-ptr-range.cpp
test/Analysis/security-syntax-checks-no-emit.c
test/Analysis/security-syntax-checks.m
test/Analysis/self-assign.cpp
test/Analysis/self-init.m
test/Analysis/shallow-mode.m
test/Analysis/simple-stream-checks.c
test/Analysis/sizeofpointer.c
test/Analysis/stack-addr-ps.c
test/Analysis/stack-addr-ps.cpp
test/Analysis/stack-block-returned.cpp
test/Analysis/stackaddrleak.c
test/Analysis/static_local.m
test/Analysis/stats.c
test/Analysis/std-c-library-functions.c
test/Analysis/std-c-library-functions.cpp
test/Analysis/stream.c
test/Analysis/string-fail.c
test/Analysis/string.c
test/Analysis/superclass.m
test/Analysis/svalbuilder-logic.c
test/Analysis/switch-case.c
test/Analysis/symbol-reaper.c
test/Analysis/taint-generic.c
test/Analysis/taint-tester.c
test/Analysis/taint-tester.cpp
test/Analysis/taint-tester.m
test/Analysis/temp-obj-dtors-cfg-output.cpp
test/Analysis/templates.cpp
test/Analysis/temporaries.cpp
test/Analysis/test-after-div-zero.c
test/Analysis/test-include-cpp.cpp
test/Analysis/test-include.c
test/Analysis/test-objc-non-nil-return-value-checker.m
test/Analysis/test-variably-modified-types.c
test/Analysis/traversal-algorithm.mm
test/Analysis/traversal-begin-end-function.c
test/Analysis/traversal-path-unification.c
test/Analysis/ubigraph-viz.cpp
test/Analysis/undef-buffers.c
test/Analysis/uninit-const.c
test/Analysis/uninit-const.cpp
test/Analysis/uninit-msg-expr.m
test/Analysis/uninit-ps-rdar6145427.m
test/Analysis/uninit-vals-ps-region.m
test/Analysis/uninit-vals-ps.c
test/Analysis/uninit-vals-union.c
test/Analysis/uninit-vals.cpp
test/Analysis/uninit-vals.m
test/Analysis/unions-region.m
test/Analysis/unions.cpp
test/Analysis/unix-api.c
test/Analysis/unix-api.cpp
test/Analysis/unix-fns.c
test/Analysis/unreachable-code-path.c
test/Analysis/unused-ivars.m
test/Analysis/valist-uninitialized.c
test/Analysis/valist-unterminated.c
test/Analysis/variadic-method-types.m
test/Analysis/vfork.c
test/Analysis/virtualcall.cpp
test/Analysis/vla.c
test/Analysis/weak-functions.c
test/lit.cfg
Quuxplusone commented 7 years ago
Below is the tabulated regression hunt for this bug...

r294758 - stage2/stage3 compare okay
r297250 - stage2/stage3 compare failure
r296003 - stage2/stage3 compare okay
r296626 - stage2/stage3 compare okay
r296938 - stage2/stage3 compare failure
r296782 - stage2/stage3 compare okay
r296860 - stage2/stage3 compare failure
r296821 - stage2/stage3 compare okay
r296841 - stage2/stage3 compare failure
r296831 - stage2/stage3 compare okay
r296836 - stage2/stage3 compare okay
r296837 - stage2/stage3 compare failure
Quuxplusone commented 7 years ago
Also the bug still exists at r298196 for current trunk...

UNRESOLVED DIFFS (stage2 vs. stage3):
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_activation.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_allocator.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_debugging.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_descriptions.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_errors.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_fake_stack.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_flags.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_globals.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_globals_win.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_interceptors.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_linux.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_mac.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_malloc_linux.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_malloc_mac.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_malloc_win.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_memory_profile.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_new_delete.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_poisoning.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_posix.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_report.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_rtl.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_stack.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_stats.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_suppressions.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_thread.cc.o
  projects/compiler-rt/lib/asan/CMakeFiles/RTAsan_dynamic.osx.dir/asan_win.cc.o
  projects/compiler-rt/lib/interception/CMakeFiles/RTInterception.osx.dir/interception_linux.cc.o
  projects/compiler-rt/lib/interception/CMakeFiles/RTInterception.osx.dir/interception_mac.cc.o
  projects/compiler-rt/lib/interception/CMakeFiles/RTInterception.osx.dir/interception_type_test.cc.o
  projects/compiler-rt/lib/interception/CMakeFiles/RTInterception.osx.dir/interception_win.cc.o
  projects/compiler-rt/lib/lsan/CMakeFiles/RTLSanCommon.osx.dir/lsan_common.cc.o
  projects/compiler-rt/lib/lsan/CMakeFiles/RTLSanCommon.osx.dir/lsan_common_linux.cc.o
  projects/compiler-rt/lib/lsan/CMakeFiles/RTLSanCommon.osx.dir/lsan_common_mac.cc.o
  projects/compiler-rt/lib/safestack/CMakeFiles/clang_rt.safestack_osx.dir/safestack.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_allocator.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_common.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_deadlock_detector1.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_deadlock_detector2.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_flag_parser.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_flags.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_libc.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_libignore.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_linux.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_linux_s390.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_mac.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_persistent_allocator.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_platform_limits_linux.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_platform_limits_posix.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_posix.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_printf.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_procmaps_common.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_procmaps_freebsd.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_procmaps_linux.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_procmaps_mac.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_stackdepot.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_stacktrace.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_stacktrace_printer.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_stoptheworld_mac.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_suppressions.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_symbolizer.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_symbolizer_libbacktrace.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_symbolizer_mac.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_symbolizer_win.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_termination.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_thread_registry.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_tls_get_addr.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.osx.dir/sanitizer_win.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sancov_flags.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_common_libcdep.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_coverage_libcdep.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_coverage_libcdep_new.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_coverage_mapping_libcdep.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_coverage_win_sections.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_linux_libcdep.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_posix_libcdep.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_stacktrace_libcdep.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_stoptheworld_linux_libcdep.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_symbolizer_libcdep.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_symbolizer_posix_libcdep.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.osx.dir/sanitizer_unwind_linux_libcdep.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoLibc.osx.dir/sanitizer_common_nolibc.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_allocator.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_common.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_deadlock_detector1.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_deadlock_detector2.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_flag_parser.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_flags.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_libc.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_libignore.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_linux.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_linux_s390.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_mac.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_persistent_allocator.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_platform_limits_linux.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_platform_limits_posix.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_posix.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_printf.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_procmaps_common.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_procmaps_freebsd.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_procmaps_linux.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_procmaps_mac.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_stackdepot.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_stacktrace.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_stacktrace_printer.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_stoptheworld_mac.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_suppressions.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_symbolizer.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_symbolizer_libbacktrace.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_symbolizer_mac.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_symbolizer_win.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_thread_registry.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_tls_get_addr.cc.o
  projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonNoTermination.osx.dir/sanitizer_win.cc.o
  projects/compiler-rt/lib/stats/CMakeFiles/clang_rt.stats_client_osx.dir/stats_client.cc.o
  projects/compiler-rt/lib/stats/CMakeFiles/clang_rt.stats_osx_dynamic.dir/stats.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_clock.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_debugging.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_external.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_fd.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_flags.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_ignoreset.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_interceptors.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_interceptors_mac.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_interface.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_interface_ann.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_interface_atomic.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_interface_java.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_libdispatch_mac.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_malloc_mac.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_md5.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_mman.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_mutex.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_mutexset.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_new_delete.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_platform_mac.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_platform_posix.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_preinit.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_report.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_rtl.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_rtl_amd64.S.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_rtl_mutex.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_rtl_proc.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_rtl_report.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_rtl_thread.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_stack_trace.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_stat.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_suppressions.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_symbolize.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/RTTsan_dynamic.osx.dir/rtl/tsan_sync.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_clock.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_debugging.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_external.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_fd.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_flags.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_ignoreset.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_interceptors.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_interceptors_mac.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_interface.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_interface_ann.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_interface_atomic.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_interface_java.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_libdispatch_mac.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_malloc_mac.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_md5.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_mman.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_mutex.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_mutexset.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_new_delete.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_platform_mac.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_platform_posix.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_preinit.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_report.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_rtl.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_rtl_amd64.S.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_rtl_mutex.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_rtl_proc.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_rtl_report.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_rtl_thread.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_stack_trace.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_stat.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_suppressions.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_symbolize.cc.o
  projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_sync.cc.o
  projects/compiler-rt/lib/ubsan/CMakeFiles/RTUbsan.osx.dir/ubsan_diag.cc.o
  projects/compiler-rt/lib/ubsan/CMakeFiles/RTUbsan.osx.dir/ubsan_flags.cc.o
  projects/compiler-rt/lib/ubsan/CMakeFiles/RTUbsan.osx.dir/ubsan_handlers.cc.o
  projects/compiler-rt/lib/ubsan/CMakeFiles/RTUbsan.osx.dir/ubsan_handlers_cxx.cc.o
  projects/compiler-rt/lib/ubsan/CMakeFiles/RTUbsan.osx.dir/ubsan_init.cc.o
  projects/compiler-rt/lib/ubsan/CMakeFiles/RTUbsan.osx.dir/ubsan_type_hash.cc.o
  projects/compiler-rt/lib/ubsan/CMakeFiles/RTUbsan.osx.dir/ubsan_type_hash_itanium.cc.o
  projects/compiler-rt/lib/ubsan/CMakeFiles/RTUbsan.osx.dir/ubsan_type_hash_win.cc.o
  projects/compiler-rt/lib/ubsan/CMakeFiles/RTUbsan.osx.dir/ubsan_value.cc.o
  projects/compiler-rt/lib/ubsan/CMakeFiles/RTUbsan_standalone.osx.dir/ubsan_init_standalone.cc.o
  projects/openmp/runtime/src/kmp_i18n_default.inc
  projects/openmp/runtime/src/kmp_i18n_id.inc
  tools/llvm-config/BuildVariables.inc
  tools/llvm-config/CMakeFiles/llvm-config.dir/llvm-config.cpp.o
  tools/llvm-readobj/CMakeFiles/llvm-readobj.dir/ELFDumper.cpp.o
Quuxplusone commented 7 years ago
As Devin has mentioned, I doubt that this commit was responsible, since it only
touches the test infrastructure for the Clang static analyzer, and doesn't
affect execution of the Clang compiler.

I'm not familiar with what stage2/stage3 are (some sort of cross-compile
setup?), but comparing DependenceAnalysis.cpp.o.stage2 and
DependenceAnalysis.cpp.o.stage3 that you uploaded, the difference is in the
DependenceAnalysis::findGCD() function, specifically the basic block placement
of the two multiply-assignment statements, relative to the R != 0 conditional.
I haven't traced through the disassembly super closely, but one appears to be
emitted as jb, the other as jnb.

01370   if (R != 0)
01371     return true; // gcd doesn't divide Delta, no dependence
01372   Q = Delta.sdiv(G);
01373   X *= Q;
01374   Y *= Q;

I'd guess that the relevant change would have something to do with CodeGen,
X86, scheduling, or an optimization somewhere.
Quuxplusone commented 7 years ago

(In reply to Dominic Chen from comment #8)

The build scripts that we have been using in the fink project for our llvm packaging (crafted by David Fang) performs a 3-stage bootstrap (ala FSF gcc) and then does a file comparison for the object files and generated headers from stage2 (produced with the stage1 tools) with those same files from stage3 (produced with the stage2 tools). The build approach can be seen at

http://fink.cvs.sourceforge.net/viewvc/fink/dists/10.9-libcxx/stable/main/finkinfo/languages/llvm39.info?revision=1.2&view=markup

I managed to produce an instance of a build with current trunk that doesn't fail the stage2/stage3 file comparison so the problem seems to be slightly non-deterministic which will make finding the offending commit difficult. Also, the exact subset of object files which differ between the stage2 and stage3 builds seems to vary slightly.

Do any of the llvm.org build bots perform a 3-stage bootstrap? And if so, do they try to compare the stage2 and stage3 object files for differences in code generation between the stage1 compiler (produced with the system compiler) and the stage2 compiler (produced with the stage1 compiler)?

Quuxplusone commented 7 years ago
Quuxplusone commented 7 years ago

Are you testing the same build script on Linux? If so can you confirm you don't see any issue there?

Can you provide more information on the Xcode version you have installed and on the host OS?

I'll try to reproduce if you can provide reproduction steps.

Quuxplusone commented 7 years ago
(In reply to Mehdi Amini from comment #11)
> Are you testing the same build script on Linux? If so can you confirm you
> don't see any issue there?
>

Unfortunately, the build in fink is a customized 3-stage bootstrap

> Can you provide more information on the Xcode version you have installed and
> on the host OS?
>
> I'll try to reproduce if you can provide reproduction steps.

Here are a couple instances of assembly differences that I observe for the
stage2 vs stage3 object files...

--- compare/tools/llvm-cov/CMakeFiles/llvm-
cov.dir/SourceCoverageViewHTML.cpp.o.dis.1   2017-03-20 20:17:34.000000000 -0400
+++ compare/tools/llvm-cov/CMakeFiles/llvm-
cov.dir/SourceCoverageViewHTML.cpp.o.dis.2   2017-03-20 20:17:34.000000000 -0400
@@ -3193,19 +3193,19 @@
 0000000000003be0       movq    __ZN4llvm19CoveragePrinterHTML14createViewFileENS_9StringRefEb(%r14), %rax ## llvm::CoveragePrinterHTML::createViewFile(llvm::StringRef, bool)
 0000000000003be3       subq    %rax, %rcx
 0000000000003be6       cmpq    $0x4, %rcx
-0000000000003bea       ja      0x3c02
+0000000000003bea       ja      0x3c05
 0000000000003bec       leaq    0x36c9(%rip), %rsi ## literal pool for: "</tr>"
 0000000000003bf3       movl    $0x5, %edx
 0000000000003bf8       movq    %rbx, %rdi
 0000000000003bfb       callq   __ZN4llvm11raw_ostream5writeEPKcm
-0000000000003c00       jmp     0x3c10
-0000000000003c02       movb    $0x3e, 0x4(%rax)
-0000000000003c06       movl    $0x72742f3c,
__ZN4llvm19CoveragePrinterHTML14createViewFileENS_9StringRefEb(%rax) ##
llvm::CoveragePrinterHTML::createViewFile(llvm::StringRef, bool)
-0000000000003c0c       addq    $0x5,
__ZN4llvm19CoveragePrinterHTML14createViewFileENS_9StringRefEb(%r14) ##
llvm::CoveragePrinterHTML::createViewFile(llvm::StringRef, bool)
-0000000000003c10       popq    %rbx
-0000000000003c11       popq    %r14
-0000000000003c13       popq    %rbp
-0000000000003c14       retq
+0000000000003c00       popq    %rbx
+0000000000003c01       popq    %r14
+0000000000003c03       popq    %rbp
+0000000000003c04       retq
+0000000000003c05       movb    $0x3e, 0x4(%rax)
+0000000000003c09       movl    $0x72742f3c,
__ZN4llvm19CoveragePrinterHTML14createViewFileENS_9StringRefEb(%rax) ##
llvm::CoveragePrinterHTML::createViewFile(llvm::StringRef, bool)
+0000000000003c0f       addq    $0x5,
__ZN4llvm19CoveragePrinterHTML14createViewFileENS_9StringRefEb(%r14) ##
llvm::CoveragePrinterHTML::createViewFile(llvm::StringRef, bool)
+0000000000003c13       jmp     0x3c00
 0000000000003c15       nopw    %cs:__ZN4llvm19CoveragePrinterHTML14createViewFileENS_9StringRefEb(%rax,%rax) ## llvm::CoveragePrinterHTML::createViewFile(llvm::StringRef, bool)
 __ZN4llvm22SourceCoverageViewHTML17renderViewDividerERNS_11raw_ostreamEj:
 0000000000003c20       pushq   %rbp

--- compare/tools/clang/tools/extra/clang-
tidy/CMakeFiles/clangTidy.dir/ClangTidyDiagnosticConsumer.cpp.o.dis.1 2017-03-
20 20:17:33.000000000 -0400
+++ compare/tools/clang/tools/extra/clang-
tidy/CMakeFiles/clangTidy.dir/ClangTidyDiagnosticConsumer.cpp.o.dis.2 2017-03-
20 20:17:34.000000000 -0400
@@ -4227,7 +4227,7 @@
 0000000000004165       movq    __ZN5clang4tidy14ClangTidyErrorC2EN4llvm9StringRefENS_7tooling10Diagnostic5LevelES3_b(%rax), %rax ## clang::tidy::ClangTidyError::ClangTidyError(llvm::StringRef, clang::tooling::Diagnostic::Level, llvm::StringRef, bool)
 0000000000004168       movq    %rax, -0x30(%rbp)
 000000000000416c       testq   %r9, %r9
-000000000000416f       je      0x44ed
+000000000000416f       je      0x44ea
 0000000000004175       imulq   $0x38, %r9, %rax
 0000000000004179       addq    %r15, %rax
 000000000000417c       movq    %rax, -0x1a8(%rbp)
@@ -4342,7 +4342,7 @@
 0000000000004355       orq     %rcx, %rdi
 0000000000004358       movq    %rdi, -0x190(%rbp)
 000000000000435f       cmpq    $0x2, %rax
-0000000000004363       jb      0x44a6
+0000000000004363       jb      0x44a3
 0000000000004369       callq   __ZN4llvm4errsEv
 000000000000436e       movq    %rax, %rbx
 0000000000004371       movq    0x10(%rbx), %rcx
@@ -4387,56 +4387,56 @@
 000000000000443e       callq   __ZN4llvm11raw_ostream5writeEPKcm
 0000000000004443       movq    0x18(%rax), %rcx
 0000000000004447       cmpq    %rcx, 0x10(%rax)
-000000000000444b       jne     0x4470
-000000000000444d       movl    $0x1, %edx
-0000000000004452       movq    %rax, %rdi
-0000000000004455       leaq    0x80fd(%rip), %rsi ## literal pool for: "\n"
-000000000000445c       callq   __ZN4llvm11raw_ostream5writeEPKcm
-0000000000004461       testb   $0x1, -0x188(%rbp)
-0000000000004468       jne     0x4480
-000000000000446a       jmp     0x448c
-000000000000446c       nopl    (%rax)
-0000000000004470       movb    $0xa,
__ZN5clang4tidy14ClangTidyErrorC2EN4llvm9StringRefENS_7tooling10Diagnostic5LevelES3_b(%rcx)
## clang::tidy::ClangTidyError::ClangTidyError(llvm::StringRef,
clang::tooling::Diagnostic::Level, llvm::StringRef, bool)
-0000000000004473       incq    0x18(%rax)
-0000000000004477       testb   $0x1, -0x188(%rbp)
-000000000000447e       je      0x448c
-0000000000004480       movq    -0x178(%rbp), %rdi
-0000000000004487       callq   __ZdlPv
-000000000000448c       movq    -0x1a0(%rbp), %rdi
-0000000000004493       andq    $-0x2, %rdi
-0000000000004497       je      0x449f
-0000000000004499       movq
__ZN5clang4tidy14ClangTidyErrorC2EN4llvm9StringRefENS_7tooling10Diagnostic5LevelES3_b(%rdi),
%rax ## clang::tidy::ClangTidyError::ClangTidyError(llvm::StringRef,
clang::tooling::Diagnostic::Level, llvm::StringRef, bool)
-000000000000449c       callq   *0x8(%rax)
-000000000000449f       movq    -0x190(%rbp), %rdi
-00000000000044a6       andq    $-0x2, %rdi
-00000000000044aa       je      0x44b2
-00000000000044ac       movq
__ZN5clang4tidy14ClangTidyErrorC2EN4llvm9StringRefENS_7tooling10Diagnostic5LevelES3_b(%rdi),
%rax ## clang::tidy::ClangTidyError::ClangTidyError(llvm::StringRef,
clang::tooling::Diagnostic::Level, llvm::StringRef, bool)
-00000000000044af       callq   *0x8(%rax)
-00000000000044b2       testb   $0x1, -0x1e8(%rbp)
-00000000000044b9       je      0x44c7
-00000000000044bb       movq    -0x1d8(%rbp), %rdi
-00000000000044c2       callq   __ZdlPv
-00000000000044c7       testb   $0x1, -0x208(%rbp)
-00000000000044ce       je      0x44dc
-00000000000044d0       movq    -0x1f8(%rbp), %rdi
-00000000000044d7       callq   __ZdlPv
-00000000000044dc       addq    $0x38, %r15
-00000000000044e0       cmpq    -0x1a8(%rbp), %r15
-00000000000044e7       jne     0x4190
-00000000000044ed       movq    ___stack_chk_guard(%rip), %rax
-00000000000044f4       movq
__ZN5clang4tidy14ClangTidyErrorC2EN4llvm9StringRefENS_7tooling10Diagnostic5LevelES3_b(%rax),
%rax ## clang::tidy::ClangTidyError::ClangTidyError(llvm::StringRef,
clang::tooling::Diagnostic::Level, llvm::StringRef, bool)
-00000000000044f7       cmpq    -0x30(%rbp), %rax
-00000000000044fb       jne     0x450f
-00000000000044fd       addq    $0x1e8, %rsp
-0000000000004504       popq    %rbx
-0000000000004505       popq    %r12
-0000000000004507       popq    %r13
-0000000000004509       popq    %r14
-000000000000450b       popq    %r15
-000000000000450d       popq    %rbp
-000000000000450e       retq
-000000000000450f       callq   ___stack_chk_fail
-0000000000004514       nopw
%cs:__ZN5clang4tidy14ClangTidyErrorC2EN4llvm9StringRefENS_7tooling10Diagnostic5LevelES3_b(%rax,%rax)
## clang::tidy::ClangTidyError::ClangTidyError(llvm::StringRef,
clang::tooling::Diagnostic::Level, llvm::StringRef, bool)
+000000000000444b       je      0x4460
+000000000000444d       movb    $0xa,
__ZN5clang4tidy14ClangTidyErrorC2EN4llvm9StringRefENS_7tooling10Diagnostic5LevelES3_b(%rcx)
## clang::tidy::ClangTidyError::ClangTidyError(llvm::StringRef,
clang::tooling::Diagnostic::Level, llvm::StringRef, bool)
+0000000000004450       incq    0x18(%rax)
+0000000000004454       testb   $0x1, -0x188(%rbp)
+000000000000445b       jne     0x447d
+000000000000445d       jmp     0x4489
+000000000000445f       nop
+0000000000004460       movl    $0x1, %edx
+0000000000004465       movq    %rax, %rdi
+0000000000004468       leaq    0x80ea(%rip), %rsi ## literal pool for: "\n"
+000000000000446f       callq   __ZN4llvm11raw_ostream5writeEPKcm
+0000000000004474       testb   $0x1, -0x188(%rbp)
+000000000000447b       je      0x4489
+000000000000447d       movq    -0x178(%rbp), %rdi
+0000000000004484       callq   __ZdlPv
+0000000000004489       movq    -0x1a0(%rbp), %rdi
+0000000000004490       andq    $-0x2, %rdi
+0000000000004494       je      0x449c
+0000000000004496       movq
__ZN5clang4tidy14ClangTidyErrorC2EN4llvm9StringRefENS_7tooling10Diagnostic5LevelES3_b(%rdi),
%rax ## clang::tidy::ClangTidyError::ClangTidyError(llvm::StringRef,
clang::tooling::Diagnostic::Level, llvm::StringRef, bool)
+0000000000004499       callq   *0x8(%rax)
+000000000000449c       movq    -0x190(%rbp), %rdi
+00000000000044a3       andq    $-0x2, %rdi
+00000000000044a7       je      0x44af
+00000000000044a9       movq
__ZN5clang4tidy14ClangTidyErrorC2EN4llvm9StringRefENS_7tooling10Diagnostic5LevelES3_b(%rdi),
%rax ## clang::tidy::ClangTidyError::ClangTidyError(llvm::StringRef,
clang::tooling::Diagnostic::Level, llvm::StringRef, bool)
+00000000000044ac       callq   *0x8(%rax)
+00000000000044af       testb   $0x1, -0x1e8(%rbp)
+00000000000044b6       je      0x44c4
+00000000000044b8       movq    -0x1d8(%rbp), %rdi
+00000000000044bf       callq   __ZdlPv
+00000000000044c4       testb   $0x1, -0x208(%rbp)
+00000000000044cb       je      0x44d9
+00000000000044cd       movq    -0x1f8(%rbp), %rdi
+00000000000044d4       callq   __ZdlPv
+00000000000044d9       addq    $0x38, %r15
+00000000000044dd       cmpq    -0x1a8(%rbp), %r15
+00000000000044e4       jne     0x4190
+00000000000044ea       movq    ___stack_chk_guard(%rip), %rax
+00000000000044f1       movq
__ZN5clang4tidy14ClangTidyErrorC2EN4llvm9StringRefENS_7tooling10Diagnostic5LevelES3_b(%rax),
%rax ## clang::tidy::ClangTidyError::ClangTidyError(llvm::StringRef,
clang::tooling::Diagnostic::Level, llvm::StringRef, bool)
+00000000000044f4       cmpq    -0x30(%rbp), %rax
+00000000000044f8       jne     0x450c
+00000000000044fa       addq    $0x1e8, %rsp
+0000000000004501       popq    %rbx
+0000000000004502       popq    %r12
+0000000000004504       popq    %r13
+0000000000004506       popq    %r14
+0000000000004508       popq    %r15
+000000000000450a       popq    %rbp
+000000000000450b       retq
+000000000000450c       callq   ___stack_chk_fail
+0000000000004511       nopw
%cs:__ZN5clang4tidy14ClangTidyErrorC2EN4llvm9StringRefENS_7tooling10Diagnostic5LevelES3_b(%rax,%rax)
## clang::tidy::ClangTidyError::ClangTidyError(llvm::StringRef,
clang::tooling::Diagnostic::Level, llvm::StringRef, bool)
 __ZN12_GLOBAL__N_127ClangTidyDiagnosticRenderer19emitIncludeLocationEN5clang14SourceLocationENS1_11PresumedLocERKNS1_13SourceManagerE:
 0000000000004520       pushq   %rbp
 0000000000004521       movq    %rsp, %rbp
Quuxplusone commented 7 years ago
I still need simple and precise reproducer steps (you sent me a diff of the
output instead, which won't help me *reproduce*).
Quuxplusone commented 7 years ago
(In reply to Mehdi Amini from comment #13)
> I still need simple and precise reproducer steps (you sent me a diff of the
> output instead, which won't help me *reproduce*).

I know but it is difficult with an issue which is non-deterministic and
produces a different subset of stage2/stage3 object file comparison failures.
Mainly I wanted to alert everyone that this issue exists since llvm trunk
branched for 4.0.0 and to be looking for stage2/stage3 file comparison failures
in the build bots.

ps I still haven't gotten a clear answer if any of the build bots do an
exhaustive file comparison for assembly differences in the object files between
the stage2 and stage3 compiler builds in a 3-stage bootstrap?
Quuxplusone commented 7 years ago

Attached ELFDumper.cpp.o.dis.diff.bz2 (381615 bytes, application/octet-stream): bzip2 of stage2/stage2 assembly diff for tools/llvm-config/CMakeFiles/llvm-config.dir/llvm-config.cpp.o

Quuxplusone commented 7 years ago

FYI, the stage2 and stage3 bootstrap cmake commands are...

cmake -DLLVM_LINK_LLVM_DYLIB:BOOL=ON -DCOMPILER_RT_ENABLE_IOS:BOOL=OFF -DLLVM_ENABLE_FFI=ON -DFFI_INCLUDE_DIR=/sw/include -DFFI_LIBRARY_DIR=/sw/lib -DLLVM_LIT_ARGS:STRING=-v -DPYTHON_EXECUTABLE:FILEPATH=/sw/bin/python2.7 -DLIBOMP_OSX_ARCHITECTURES=x86_64;i386 -DLLVM_ENABLE_ASSERTIONS:BOOL=OFF -DCMAKE_OSX_SYSROOT:STRING=/ -DCMAKE_OSX_DEPLOYMENT_TARGET:STRING= -DLLVM_TARGETS_TO_BUILD=X86;PowerPC;ARM -DCMAKE_INSTALL_PREFIX:PATH=/sw/opt/llvm-5.0 -DCMAKE_BUILD_TYPE:STRING=Release -DLIBOMP_CFLAGS=-cxx-isystem /sw/src/fink.build/llvm50-5.0.0-1/llvm-5.0.0.src/../libcxx-5.0.0.src/include -DCMAKE_CXX_FLAGS=-std=c++11 -stdlib=libc++ -cxx-isystem /sw/src/fink.build/llvm50-5.0.0-1/llvm-5.0.0.src/../libcxx-5.0.0.src/include -L/sw/src/fink.build/llvm50-5.0.0-1/llvm-5.0.0.src/../build/last-libcxx/lib ../../llvm-5.0.0.src

Quuxplusone commented 7 years ago
(In reply to Jack Howarth from comment #14)
> (In reply to Mehdi Amini from comment #13)
> > I still need simple and precise reproducer steps (you sent me a diff of the
> > output instead, which won't help me *reproduce*).
>
> I know but it is difficult with an issue which is non-deterministic and
> produces a different subset of stage2/stage3 object file comparison
> failures.

Shouldn't be that difficult, I'll run this in a loop till it fails.

> ps I still haven't gotten a clear answer if any of the build bots do an
> exhaustive file comparison for assembly differences in the object files
> between the stage2 and stage3 compiler builds in a 3-stage bootstrap?

I don't have this answer in the absolute. I'm pretty sure it is not the case
for the MacOS testing and it is on our to-do list for some time.
Quuxplusone commented 7 years ago

(In reply to Mehdi Amini from comment #17)

Are you building with some variation of the 3-Stage Non-Determinism build described at http://llvm.org/docs/AdvancedBuilds.html so that the stage2 and stage3 binaries can be compared?

Quuxplusone commented 7 years ago

I haven't done anything yet, my test machine died and I reinstalled it this morning.

It'd help to have a script to run to reproduce though.

Quuxplusone commented 7 years ago
(In reply to Mehdi Amini from comment #19)
> I haven't done anything yet, my test machine died and I reinstalled it this
> morning.
>
> It'd help to have a script to run to reproduce though.

I would expect the most straightforward approach to achieving that would be to
adjust the existing cmake/caches/3-stage.cmake from clang to allow for the 3-
stage bootstrap on darwin. I assume that the current permutation has only been
tested on ubuntu linux...

https://reviews.llvm.org/D17653

although oddly I don't see that one listed on http://lab.llvm.org:8011/builders
for some reason.
Quuxplusone commented 7 years ago

I think the most straightforward way is just a list of commands to run that would work on any platform.

Quuxplusone commented 7 years ago
(In reply to Mehdi Amini from comment #21)
> I think the most straightforward way is just a list of commands to run that
> would work on any platform.

Unfortunately such a simple set of commands doesn't exist. In the fink project
we have been used a highly customized build script to achieve a 3 stage
bootstrap...

http://fink.cvs.sourceforge.net/viewvc/fink/dists/10.9-libcxx/stable/main/finkinfo/languages/llvm39.info?revision=1.2&view=markup

that is tethered to the fink packaging system. Looking at....

https://llvm.org/svn/llvm-project/zorg/trunk/buildbot/osuosl/master/config/builders.py

I see that the clang-3stage-ubuntu section is commented out so it would seem
that no else one is currently doing 3-stage bootstraps (on the build bots
anyway) to check for non-deterministic code generation.
Quuxplusone commented 7 years ago
(In reply to Jack Howarth from comment #22)
> (In reply to Mehdi Amini from comment #21)
> > I think the most straightforward way is just a list of commands to run that
> > would work on any platform.
>
> Unfortunately such a simple set of commands doesn't exist. In the fink
> project we have been used a highly customized build script to achieve a 3
> stage bootstrap...

Sure, but unless there *is* a set of commands to run, it is hard to even figure
*if* the bug isn't in the custom scripts.
Quuxplusone commented 7 years ago

(In reply to Mehdi Amini from comment #23)

A brute force regression hunting back through the llvm svn commits shows that the non-deterministic 3-stage bootstrap failures become increasingly more latent the further back you go towards the offending commit. I've seen the issue as far back as r296825. So we may have a situation where the actual offending commit leaves the issue entirely latent.

Currently trunk has the issue well-exposeed those so it shouldn't be hard to reproduce there (but that won't do much to identifying the cause). Currently tools/llvm-readobj/CMakeFiles/llvm-readobj.dir/ELFDumper.cpp.o seems to be a particular hot spot for non-deterministic compilation between the stage1 and stage2 compilers.

Quuxplusone commented 7 years ago
I have also reproduced the non-determinism in the 3-stage bootstrap of current
trunk on a second machine, a MacPro 4,1 running Sierra.

--- compare/tools/llvm-cov/CMakeFiles/llvm-
cov.dir/SourceCoverageViewHTML.cpp.o.dis.1   2017-03-26 14:03:40.000000000 -0400
+++ compare/tools/llvm-cov/CMakeFiles/llvm-
cov.dir/SourceCoverageViewHTML.cpp.o.dis.2   2017-03-26 14:03:40.000000000 -0400
@@ -3181,19 +3181,19 @@
 0000000000003bb0       movq    __ZN4llvm19CoveragePrinterHTML14createViewFileENS_9StringRefEb(%r14), %rax ## llvm::CoveragePrinterHTML::createViewFile(llvm::StringRef, bool)
 0000000000003bb3       subq    %rax, %rcx
 0000000000003bb6       cmpq    $0x4, %rcx
-0000000000003bba       ja      0x3bd5
+0000000000003bba       ja      0x3bd2
 0000000000003bbc       leaq    0x3679(%rip), %rsi ## literal pool for: "</tr>"
 0000000000003bc3       movl    $0x5, %edx
 0000000000003bc8       movq    %rbx, %rdi
 0000000000003bcb       callq   __ZN4llvm11raw_ostream5writeEPKcm
-0000000000003bd0       popq    %rbx
-0000000000003bd1       popq    %r14
-0000000000003bd3       popq    %rbp
-0000000000003bd4       retq
-0000000000003bd5       movb    $0x3e, 0x4(%rax)
-0000000000003bd9       movl    $0x72742f3c,
__ZN4llvm19CoveragePrinterHTML14createViewFileENS_9StringRefEb(%rax) ##
llvm::CoveragePrinterHTML::createViewFile(llvm::StringRef, bool)
-0000000000003bdf       addq    $0x5,
__ZN4llvm19CoveragePrinterHTML14createViewFileENS_9StringRefEb(%r14) ##
llvm::CoveragePrinterHTML::createViewFile(llvm::StringRef, bool)
-0000000000003be3       jmp     0x3bd0
+0000000000003bd0       jmp     0x3be0
+0000000000003bd2       movb    $0x3e, 0x4(%rax)
+0000000000003bd6       movl    $0x72742f3c,
__ZN4llvm19CoveragePrinterHTML14createViewFileENS_9StringRefEb(%rax) ##
llvm::CoveragePrinterHTML::createViewFile(llvm::StringRef, bool)
+0000000000003bdc       addq    $0x5,
__ZN4llvm19CoveragePrinterHTML14createViewFileENS_9StringRefEb(%r14) ##
llvm::CoveragePrinterHTML::createViewFile(llvm::StringRef, bool)
+0000000000003be0       popq    %rbx
+0000000000003be1       popq    %r14
+0000000000003be3       popq    %rbp
+0000000000003be4       retq
 0000000000003be5       nopw    %cs:__ZN4llvm19CoveragePrinterHTML14createViewFileENS_9StringRefEb(%rax,%rax) ## llvm::CoveragePrinterHTML::createViewFile(llvm::StringRef, bool)
 __ZN4llvm22SourceCoverageViewHTML17renderViewDividerERNS_11raw_ostreamEj:
 0000000000003bf0       pushq   %rbp
Quuxplusone commented 7 years ago
(In reply to Mehdi Amini from comment #23)

I should also note that, for the object files which fail the stage2/stage3 file
comparison in the 3-stage bootstrap on darwin, I also see variation in those
same stage2 files between repeated bootstraps for the same llvm svn revision.
So you should be able to detect this non-deterministic behavior from the 2-
stage Apple bootstrap described in
http://llvm.org/docs/AdvancedBuilds.html#id3 under "Apple Clang Builds (A More
Complex Bootstrap)".

Unfortunately the documentation provided there seems to be incomplete on how to
configure such a build from scratch so someone at Apple who understands how to
properly run one with...

$ cmake -G Ninja -C <path to clang>/cmake/caches/Apple-stage1.cmake <path to
source>
$ ninja stage2-distribution

will have to try to reproduce the bug.
Quuxplusone commented 7 years ago
A basic stage-2 build is trivial using only these options: -
DCMAKE_C_COMPILER=path/to/stage-1/bin/clang and -
DCMAKE_CXX_COMPILER=path/to/stage-1/bin/clang++
The cmake cache files aren't needed.
Quuxplusone commented 7 years ago
(In reply to Mehdi Amini from comment #27)
Okay. I can reproduce the non-determinitism with a simple cmake build of
llvm/clang/clang-tools-extra/compiler-rt/libcxx with...

 cmake -DLLVM_LINK_LLVM_DYLIB:BOOL=ON -DCOMPILER_RT_ENABLE_IOS:BOOL=OFF -DPYTHON_EXECUTABLE:FILEPATH=/opt/local/bin/python2.7 -DLLVM_ENABLE_ASSERTIONS:BOOL=OFF -DCMAKE_OSX_SYSROOT:STRING=/ -DCMAKE_OSX_DEPLOYMENT_TARGET:STRING= -DLLVM_TARGETS_TO_BUILD="X86;PowerPC;ARM" -DCMAKE_BUILD_TYPE:STRING=Release /Users/howarth/2stage/llvm-5.0.0.src

for stage1 and

cmake -DLLVM_LINK_LLVM_DYLIB:BOOL=ON -DCOMPILER_RT_ENABLE_IOS:BOOL=OFF -
DPYTHON_EXECUTABLE:FILEPATH=/opt/local/bin/python2.7 -
DLLVM_ENABLE_ASSERTIONS:BOOL=OFF -DCMAKE_OSX_SYSROOT:STRING=/ -
DCMAKE_OSX_DEPLOYMENT_TARGET:STRING= -DLLVM_TARGETS_TO_BUILD="X86;PowerPC;ARM" -
DCMAKE_BUILD_TYPE:STRING=Release -
DCMAKE_C_COMPILER=/Users/howarth/2stage/build/bin/clang -
DCMAKE_CXX_COMPILER=/Users/howarth/2stage/build/bin/clang++
/Users/howarth/2stage/llvm-5.0.0.src

for stage2. Repeating these two builds produces the following assembly
differences between each the two different stage2 builds...

--- stage2_ELFDumper.cpp.o.disasm   2017-03-27 12:57:48.000000000 -0400
+++ repeat_stage2_ELFDumper.cpp.o.disasm    2017-03-27 12:59:27.000000000 -0400
@@ -1,4 +1,4 @@
-/Users/howarth/2stage/build_stage2/tools/llvm-readobj/CMakeFiles/llvm-readobj.dir/ELFDumper.cpp.o:
+/Users/howarth/2stage/repeat_build_stage2/tools/llvm-readobj/CMakeFiles/llvm-readobj.dir/ELFDumper.cpp.o:
 (__TEXT,__text) section
 __ZN4llvm3ARM5EHABI13OpcodeDecoder15Decode_00xxxxxxEPKhRj:
 0000000000000000   pushq   %rbp
@@ -90384,28 +90384,28 @@
 0000000000062eb9   movq    0x10(%rbx), %rax
 0000000000062ebd   subq    %rdi, %rax
 0000000000062ec0   cmpq    $0x1, %rax
-0000000000062ec4   ja  0x62edf
+0000000000062ec4   ja  0x62f11
 0000000000062ec6   leaq    0x26aa8(%rip), %rsi ## literal pool for: ": "
 0000000000062ecd   movl    $0x2, %edx
 0000000000062ed2   movq    %rbx, %rdi
 0000000000062ed5   callq   __ZN4llvm11raw_ostream5writeEPKcm
 0000000000062eda   movq    %rax, %rbx
-0000000000062edd   jmp 0x62ee9
-0000000000062edf   movw    $0x203a,
__ZN4llvm3ARM5EHABI13OpcodeDecoder15Decode_00xxxxxxEPKhRj(%rdi) ##
llvm::ARM::EHABI::OpcodeDecoder::Decode_00xxxxxx(unsigned char const*, unsigned
int&)
-0000000000062ee4   addq    $0x2, 0x18(%rbx)
-0000000000062ee9   movq    -0x38(%rbp), %rax
-0000000000062eed   movq    %rax, -0x40(%rbp)
-0000000000062ef1   leaq    -0x40(%rbp), %rsi
-0000000000062ef5   movq    %rbx, %rdi
-0000000000062ef8   callq   __ZN4llvmlsERNS_11raw_ostreamERKNS_9HexNumberE
-0000000000062efd   movq    0x18(%rax), %rcx
-0000000000062f01   cmpq    %rcx, 0x10(%rax)
-0000000000062f05   jne 0x62f1d
-0000000000062f07   leaq    0x25be0(%rip), %rsi ## literal pool for: "\n"
-0000000000062f0e   movl    $0x1, %edx
-0000000000062f13   movq    %rax, %rdi
-0000000000062f16   callq   __ZN4llvm11raw_ostream5writeEPKcm
-0000000000062f1b   jmp 0x62f24
+0000000000062edd   movq    -0x38(%rbp), %rax
+0000000000062ee1   movq    %rax, -0x40(%rbp)
+0000000000062ee5   leaq    -0x40(%rbp), %rsi
+0000000000062ee9   movq    %rbx, %rdi
+0000000000062eec   callq   __ZN4llvmlsERNS_11raw_ostreamERKNS_9HexNumberE
+0000000000062ef1   movq    0x18(%rax), %rcx
+0000000000062ef5   cmpq    %rcx, 0x10(%rax)
+0000000000062ef9   jne 0x62f1d
+0000000000062efb   leaq    0x25bec(%rip), %rsi ## literal pool for: "\n"
+0000000000062f02   movl    $0x1, %edx
+0000000000062f07   movq    %rax, %rdi
+0000000000062f0a   callq   __ZN4llvm11raw_ostream5writeEPKcm
+0000000000062f0f   jmp 0x62f24
+0000000000062f11   movw    $0x203a,
__ZN4llvm3ARM5EHABI13OpcodeDecoder15Decode_00xxxxxxEPKhRj(%rdi) ##
llvm::ARM::EHABI::OpcodeDecoder::Decode_00xxxxxx(unsigned char const*, unsigned
int&)
+0000000000062f16   addq    $0x2, 0x18(%rbx)
+0000000000062f1b   jmp 0x62edd
 0000000000062f1d   movb    $0xa, __ZN4llvm3ARM5EHABI13OpcodeDecoder15Decode_00xxxxxxEPKhRj(%rcx) ## llvm::ARM::EHABI::OpcodeDecoder::Decode_00xxxxxx(unsigned char const*, unsigned int&)
 0000000000062f20   incq    0x18(%rax)
 0000000000062f24   addq    $0x18, %rsp

So this shouldn't be hard for you to reproduce.
Quuxplusone commented 7 years ago
It took several retries of a stage2 build but I was also able to produce the
following code generation diff using the same stage1 compiler build...

--- stage2_SourceCoverageViewHTML.cpp.o.disasm  2017-03-27 12:32:30.000000000 -
0400
+++ retry_stage2_SourceCoverageViewHTML.cpp.o.disasm    2017-03-27
14:59:41.000000000 -0400
@@ -1,4 +1,4 @@
-/Users/howarth/2stage/build_stage2/tools/llvm-cov/CMakeFiles/llvm-cov.dir/SourceCoverageViewHTML.cpp.o:
+/Users/howarth/2stage/retry_build_stage2/tools/llvm-cov/CMakeFiles/llvm-cov.dir/SourceCoverageViewHTML.cpp.o:
 (__TEXT,__text) section
 __ZN4llvm19CoveragePrinterHTML14createViewFileENS_9StringRefEb:
 0000000000000000   pushq   %rbp
@@ -3182,19 +3182,19 @@
 0000000000003bb0   movq    __ZN4llvm19CoveragePrinterHTML14createViewFileENS_9StringRefEb(%r14), %rax ## llvm::CoveragePrinterHTML::createViewFile(llvm::StringRef, bool)
 0000000000003bb3   subq    %rax, %rcx
 0000000000003bb6   cmpq    $0x4, %rcx
-0000000000003bba   ja  0x3bd2
+0000000000003bba   ja  0x3bd5
 0000000000003bbc   leaq    0x3679(%rip), %rsi ## literal pool for: "</tr>"
 0000000000003bc3   movl    $0x5, %edx
 0000000000003bc8   movq    %rbx, %rdi
 0000000000003bcb   callq   __ZN4llvm11raw_ostream5writeEPKcm
-0000000000003bd0   jmp 0x3be0
-0000000000003bd2   movb    $0x3e, 0x4(%rax)
-0000000000003bd6   movl    $0x72742f3c,
__ZN4llvm19CoveragePrinterHTML14createViewFileENS_9StringRefEb(%rax) ##
llvm::CoveragePrinterHTML::createViewFile(llvm::StringRef, bool)
-0000000000003bdc   addq    $0x5,
__ZN4llvm19CoveragePrinterHTML14createViewFileENS_9StringRefEb(%r14) ##
llvm::CoveragePrinterHTML::createViewFile(llvm::StringRef, bool)
-0000000000003be0   popq    %rbx
-0000000000003be1   popq    %r14
-0000000000003be3   popq    %rbp
-0000000000003be4   retq
+0000000000003bd0   popq    %rbx
+0000000000003bd1   popq    %r14
+0000000000003bd3   popq    %rbp
+0000000000003bd4   retq
+0000000000003bd5   movb    $0x3e, 0x4(%rax)
+0000000000003bd9   movl    $0x72742f3c,
__ZN4llvm19CoveragePrinterHTML14createViewFileENS_9StringRefEb(%rax) ##
llvm::CoveragePrinterHTML::createViewFile(llvm::StringRef, bool)
+0000000000003bdf   addq    $0x5,
__ZN4llvm19CoveragePrinterHTML14createViewFileENS_9StringRefEb(%r14) ##
llvm::CoveragePrinterHTML::createViewFile(llvm::StringRef, bool)
+0000000000003be3   jmp 0x3bd0
 0000000000003be5   nopw    %cs:__ZN4llvm19CoveragePrinterHTML14createViewFileENS_9StringRefEb(%rax,%rax) ## llvm::CoveragePrinterHTML::createViewFile(llvm::StringRef, bool)
 __ZN4llvm22SourceCoverageViewHTML17renderViewDividerERNS_11raw_ostreamEj:
 0000000000003bf0   pushq   %rbp
Quuxplusone commented 7 years ago

Attached SourceCoverageViewHTML.ii.bz2 (201545 bytes, application/octet-stream): bzip2 compressed preprocessed source for tools/llvm-cov/CMakeFiles/llvm-cov.dir/SourceCoverageViewHTML.cpp

Quuxplusone commented 7 years ago
Repeated compiles of the attached SourceCoverageViewHTML.ii with the clang
compiler built in stage1 using the command...

/Users/howarth/2stage/build/bin/clang++ -DGTEST_HAS_RTTI=0 -
DLLVM_BUILD_GLOBAL_ISEL -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -
D__STDC_LIMIT_MACROS -fPIC -fvisibility-inlines-hidden -Werror=date-time -
std=c++11 -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-
field-initializers -pedantic -Wno-long-long -Wcovered-switch-default -Wnon-
virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion -O3 -DNDEBUG -fno-
exceptions -fno-rtti -o SourceCoverageViewHTML.cpp.o -c
SourceCoverageViewHTML.ii --save-temps

will eventually yield the assembly difference of...

--- SourceCoverageViewHTML.s.sav    2017-03-27 15:23:30.000000000 -0400
+++ SourceCoverageViewHTML.s    2017-03-27 15:24:13.000000000 -0400
@@ -3976,16 +3976,16 @@
    movl    $5, %edx
    movq    %rbx, %rdi
    callq   __ZN4llvm11raw_ostream5writeEPKcm
-   jmp LBB14_7
-LBB14_6:
-   movb    $62, 4(%rax)
-   movl    $1920216892, (%rax)     ## imm = 0x72742F3C
-   addq    $5, (%r14)
 LBB14_7:                                ## %_ZN4llvm11raw_ostreamlsEPKc.exit4
    popq    %rbx
    popq    %r14
    popq    %rbp
    retq
+LBB14_6:
+   movb    $62, 4(%rax)
+   movl    $1920216892, (%rax)     ## imm = 0x72742F3C
+   addq    $5, (%r14)
+   jmp LBB14_7
    .cfi_endproc

    .globl  __ZN4llvm22SourceCoverageViewHTML17renderViewDividerERNS_11raw_ostreamEj

clearly showing the llvm trunk has become non-deterministic since branching for
the 4.0.0 release.
Quuxplusone commented 7 years ago

The assembly generated from the SourceCoverageViewHTML.ii test case by the stage1 compiler exhibits the non-determinism for the -O3 and -O2 optimization levels but not for -O1 or -Os.

Quuxplusone commented 7 years ago

Attached ClangTidyDiagnosticConsumer.ii.bz2 (385701 bytes, application/octet-stream): bzip2 compressed preprocessed source for tools/clang/tools/extra/clang-tidy/ClangTidyDiagnosticConsumer.cpp

Quuxplusone commented 7 years ago
Repeated compiles of the attached SourceCoverageViewHTML.ii with the clang
compiler built in stage1 using the command...

/Users/howarth/build/stage1/bin/clang++ -DCLANG_ENABLE_ARCMT -
DCLANG_ENABLE_OBJC_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -DGTEST_HAS_RTTI=0 -
DLLVM_BUILD_GLOBAL_ISEL -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -
D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -fPIC -fvisibility-inlines-hidden -
Werror=date-time -std=c++11 -Wall -W -Wno-unused-parameter -Wwrite-strings -
Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wcovered-
switch-default -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion
-fno-common -Woverloaded-virtual -Wno-nested-anon-types -O3 -DNDEBUG -fno-
exceptions -fno-rtti -o
CMakeFiles/clangTidy.dir/ClangTidyDiagnosticConsumer.cpp.o -c
ClangTidyDiagnosticConsumer.ii --save-temps

will eventually yield the assembly difference of...

--- ClangTidyDiagnosticConsumer.s.sav   2017-03-28 10:47:04.000000000 -0400
+++ ClangTidyDiagnosticConsumer.s   2017-03-28 10:47:52.000000000 -0400
@@ -6162,19 +6162,19 @@
    callq   __ZN4llvm11raw_ostream5writeEPKcm
    movq    24(%rax), %rcx
    cmpq    %rcx, 16(%rax)
-   jne LBB47_22
-## BB#21:                               ##   in Loop: Header=BB47_2 Depth=1
-   movl    $1, %edx
-   movq    %rax, %rdi
-   leaq    L_.str.15(%rip), %rsi
-   callq   __ZN4llvm11raw_ostream5writeEPKcm
+   je  LBB47_21
+## BB#22:                               ##   in Loop: Header=BB47_2 Depth=1
+   movb    $10, (%rcx)
+   incq    24(%rax)
    testb   $1, -392(%rbp)
    jne LBB47_24
    jmp LBB47_25
    .p2align    4, 0x90
-LBB47_22:                               ##   in Loop: Header=BB47_2 Depth=1
-   movb    $10, (%rcx)
-   incq    24(%rax)
+LBB47_21:                               ##   in Loop: Header=BB47_2 Depth=1
+   movl    $1, %edx
+   movq    %rax, %rdi
+   leaq    L_.str.15(%rip), %rsi
+   callq   __ZN4llvm11raw_ostream5writeEPKcm
    testb   $1, -392(%rbp)
    je  LBB47_25
 LBB47_24:                               ##   in Loop: Header=BB47_2 Depth=1

which is another reproducer for the non-deterministic behavior of llvm trunk.
Quuxplusone commented 7 years ago

Attached ELFDumper.ii.bz2 (247172 bytes, application/octet-stream): bzip2 compressed preprocessed source for tools/llvm-readobj/ELFDumper.cpp

Quuxplusone commented 7 years ago
Repeated compiles of the attached ELFDumper.ii with the clang compiler built in
stage1 using the command...

/Users/howarth/build/stage1/bin/clang++ -DGTEST_HAS_RTTI=0 -
DLLVM_BUILD_GLOBAL_ISEL -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -
D__STDC_LIMIT_MACROS -fPIC -fvisibility-inlines-hidden -Werror=date-time -
std=c++11 -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-
field-initializers -pedantic -Wno-long-long -Wcovered-switch-default -Wnon-
virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion -O3 -DNDEBUG -fno-
exceptions -fno-rtti -o ELFDumper.cpp.o -c ELFDumper.ii --save-temps

will eventually yield the assembly difference of...

--- ELFDumper.s.sav 2017-03-28 10:55:14.000000000 -0400
+++ ELFDumper.s 2017-03-28 10:56:43.000000000 -0400
@@ -108520,33 +108520,33 @@
    leaq    -88(%rbp), %rdi
    callq   __ZN4llvm18raw_string_ostreamD1Ev
    testb   $1, -144(%rbp)
-   je  LBB356_36
-## BB#35:
+   je  LBB356_35
+## BB#41:
    movq    -128(%rbp), %rdi
    callq   __ZdlPv
-LBB356_36:                              ##
%_ZN4llvm9to_stringINS_15FormattedNumberEEEKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEERKT_.exit77
+LBB356_35:                              ##
%_ZN4llvm9to_stringINS_15FormattedNumberEEEKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEERKT_.exit77
    movq    %rbx, -280(%rbp)        ## 8-byte Spill
    leaq    -320(%rbp), %rdi
    leaq    -184(%rbp), %rsi
    callq   __ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEaSERKS5_
    testb   $1, -184(%rbp)
-   je  LBB356_38
-## BB#37:
+   je  LBB356_37
+## BB#36:
    movq    -168(%rbp), %rdi
    callq   __ZdlPv
-LBB356_38:                              ##
%_ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEED1Ev.exit78
+LBB356_37:                              ##
%_ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEED1Ev.exit78
    leaq    16(%rbp), %rax
    movq    16(%rax), %rbx
    movq    -160(%rbp), %rax        ## 8-byte Reload
    testq   %rax, %rax
    je  LBB356_44
-## BB#39:                               ##
%_ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEED1Ev.exit78
+## BB#38:                               ##
%_ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEED1Ev.exit78
    testb   %r13b, %r13b
    je  LBB356_44
-## BB#40:
+## BB#39:
    testq   %rbx, %rbx
    js  LBB356_42
-## BB#41:
+## BB#40:
    leaq    L_.str.740(%rip), %rsi
    jmp LBB356_43
 LBB356_42:

which is another reproducer for the non-deterministic behavior of llvm trunk.
Quuxplusone commented 7 years ago
(In reply to Jack Howarth from comment #34)

The code permutation...

>   jne LBB47_22
>  ## BB#21:                               ##   in Loop: Header=BB47_2 Depth=1
>   movl    $1, %edx
>   movq    %rax, %rdi
>   leaq    L_.str.15(%rip), %rsi
>   callq   __ZN4llvm11raw_ostream5writeEPKcm
>   testb   $1, -392(%rbp)
>   jne LBB47_24
>   jmp LBB47_25
>   .p2align    4, 0x90
>  LBB47_22:                               ##   in Loop: Header=BB47_2 Depth=1
>   movb    $10, (%rcx)
>   incq    24(%rax)
>   testb   $1, -392(%rbp)
>   je  LBB47_25
>  LBB47_24:                               ##   in Loop: Header=BB47_2 Depth=1
>

generated for the compilation of ClangTidyDiagnosticConsumer.ii under trunk
matches that generated by the llvm 4.0.0 compiler. So I assume this is the
'correct' codegen.
Quuxplusone commented 7 years ago
The non-deterministic behavior for the attached ClangTidyDiagnosticConsumer.ii,
ELFDumper.ii and SourceCoverageViewHTML.ii test cases is reproducible under
Ubuntu 17.04 beta 2 x86_64 linux with the March 30th Ubuntu nightly build from
http://apt.llvm.org/ for clang-5.0 installed. The test cases were compiled
with...

clang++-5.0 -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_OBJC_REWRITER -
DCLANG_ENABLE_STATIC_ANALYZER -DGTEST_HAS_RTTI=0 -DLLVM_BUILD_GLOBAL_ISEL -
D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -
D__STDC_LIMIT_MACROS -fPIC -fvisibility-inlines-hidden -Werror=date-time -
std=c++11 -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-
field-initializers -pedantic -Wno-long-long -Wcovered-switch-default -Wnon-
virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion -fno-common -
Woverloaded-virtual -Wno-nested-anon-types -O3 -DNDEBUG -fno-exceptions -fno-
rtti -o ClangTidyDiagnosticConsumer.cpp.o -c ClangTidyDiagnosticConsumer.ii -
save-temps -fblocks

clang++-5.0 -DGTEST_HAS_RTTI=0 -DLLVM_BUILD_GLOBAL_ISEL -
D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -fPIC -
fvisibility-inlines-hidden -Werror=date-time -std=c++11 -Wall -W -Wno-unused-
parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -
Wno-long-long -Wcovered-switch-default -Wnon-virtual-dtor -Wdelete-non-virtual-
dtor -Wstring-conversion -O3 -DNDEBUG -fno-exceptions -fno-rtti -o
ELFDumper.cpp.o -c ELFDumper.ii --save-temps -fblocks

clang++-5.0 -DGTEST_HAS_RTTI=0 -DLLVM_BUILD_GLOBAL_ISEL -
D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -fPIC -
fvisibility-inlines-hidden -Werror=date-time -std=c++11 -Wall -W -Wno-unused-
parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -
Wno-long-long -Wcovered-switch-default -Wnon-virtual-dtor -Wdelete-non-virtual-
dtor -Wstring-conversion -O3 -DNDEBUG -fno-exceptions -fno-rtti -o
SourceCoverageViewHTML.cpp.o -c SourceCoverageViewHTML.ii --save-temps  -fblocks

with attached assembly variations produced between repeated compiles of the
test cases.
Quuxplusone commented 7 years ago

Attached ELFDumper.s.diff.linux (4863 bytes, text/plain): asembly non-determinism for ELFDumper.ii on x86_64 linux

Quuxplusone commented 7 years ago

Thanks for the debug and getting the preprocessed reproducer, that should be enough to help figuring out.

Quuxplusone commented 7 years ago

Attached SourceCoverageViewHTML.s.diff.linux (784 bytes, text/plain): asembly non-determinism for SourceCoverageViewHTML.ii on x86_64 linux

Quuxplusone commented 7 years ago

Attached ClangTidyDiagnosticConsumer.s.diff.linux (1052 bytes, text/plain): asembly non-determinism for ClangTidyDiagnosticConsumer.ii on x86_64 linux

Quuxplusone commented 7 years ago
This is caused by iterating a dense map.

Author: iteratee
Date: Thu Mar  2 19:00:22 2017
New Revision: 296845

URL: http://llvm.org/viewvc/llvm-project?rev=296845&view=rev
Log:
CodeGen: BlockPlacement: Precompute layout for chains of triangles.

For chains of triangles with small join blocks that can be tail duplicated, a
simple calculation of probabilities is insufficient. Tail duplication
can be profitable in 3 different ways for these cases:

1) The post-dominators marked 50% are actually taken 56% (This shrinks with
  longer chains)
2) The chains are statically correlated. Branch probabilities have a very
  U-shaped distribution.
  [http://nrs.harvard.edu/urn-3:HUL.InstRepos:24015805]
  If the branches in a chain are likely to be from the same side of the
  distribution as their predecessor, but are independent at runtime, this
  transformation is profitable. (Because the cost of being wrong is a small
  fixed cost, unlike the standard triangle layout where the cost of being
  wrong scales with the # of triangles.)
3) The chains are dynamically correlated. If the probability that a previous
  branch was taken positively influences whether the next branch will be
  taken
We believe that 2 and 3 are common enough to justify the small margin in 1.

The code pre-scans a function's CFG to identify this pattern and marks the edges
so that the standard layout algorithm can use the computed results.

Modified:
   llvm/trunk/lib/CodeGen/MachineBlockPlacement.cpp
   llvm/trunk/test/CodeGen/PowerPC/tail-dup-layout.ll
   llvm/trunk/test/CodeGen/X86/cmovcmov.ll
Quuxplusone commented 7 years ago
(In reply to Gerolf Hoflehner from comment #43)
> This is caused by iterating a dense map.
>
> ...
>
> Modified:
>    llvm/trunk/lib/CodeGen/MachineBlockPlacement.cpp
>    llvm/trunk/test/CodeGen/PowerPC/tail-dup-layout.ll
>    llvm/trunk/test/CodeGen/X86/cmovcmov.ll

From what I could see, the output of SelectionDAG (or right after Expand-Pseudo-
ISEL...) is showing the diff. Isn't MBP running a lot later?
Quuxplusone commented 7 years ago
(In reply to Gerolf Hoflehner from comment #43)
> This is caused by iterating a dense map.
>
> Author: iteratee
> Date: Thu Mar  2 19:00:22 2017
> New Revision: 296845
>

A brute force regression hunt on x86_64 darwin produced at least one instance
of the non-deterministic behavior at r296825 which would pre-date that
particular commit.
Quuxplusone commented 7 years ago

@Mehdi: Could you be more specific? What difference did you see where? If you used a stage 2 compiler the differences could be anywhere.

@Jack: I can't reproduce your result. Could you double check? The block ordering issue I can reproduce and is consistent with the differences I see: the same instructions get generated, but the blocks are permuted. Thus some branch instructions and labsls could be different. I only looked at your ELFDumper.ii example. Perhaps there is a second problem shown by one of your other test cases. I used the ELFDumper.ii. Also, I used a stage 1 compiler in my (final) bisection. My build compiler is a stable release from last year.

Quuxplusone commented 7 years ago
I've been looking at a "print-after-all" output and the first diff I saw
occurred in the fist dump of machine code (i.e. the IR was the same), it always
looked like this pattern:

1000,1003c1000,1004
<       %vreg67<def> = MOV32rm %vreg30, 1, %noreg, 16, %noreg;
mem:LD4[%sunkaddr50](align=8)(tbaa=!65) GR32:%vreg67 GR64:%vreg30
<       %vreg68<def> = SUBREG_TO_REG 0, %vreg67<kill>, sub_32bit; GR64:%vreg68
GR32:%vreg67
<       %vreg69<def,tied1> = SHL64ri %vreg68<tied0>, 5, %EFLAGS<imp-def,dead>;
GR64:%vreg69,%vreg68
<       %vreg70<def,tied1> = ADD64rm %vreg69<tied0>, %vreg30, 1, %noreg, 0,
%noreg, %EFLAGS<imp-def,dead>; mem:LD8[%15](tbaa=!65)
GR64:%vreg70,%vreg69,%vreg30
---
>       %vreg67<def> = MOV64rm %vreg30, 1, %noreg, 0, %noreg;
mem:LD8[%15](tbaa=!65) GR64:%vreg67,%vreg30
>       %vreg68<def> = MOV32rm %vreg30, 1, %noreg, 16, %noreg;
mem:LD4[%sunkaddr50](align=8)(tbaa=!65) GR32:%vreg68 GR64:%vreg30
>       %vreg69<def> = SUBREG_TO_REG 0, %vreg68<kill>, sub_32bit; GR64:%vreg69
GR32:%vreg68
>       %vreg70<def,tied1> = SHL64ri %vreg69<tied0>, 5, %EFLAGS<imp-def,dead>;
GR64:%vreg70,%vreg69
>       %vreg71<def,tied1> = ADD64rr %vreg67<tied0>, %vreg70<kill>, %EFLAGS<imp-
def,dead>; GR64:%vreg71,%vreg67,%vreg70
Quuxplusone commented 7 years ago
(In reply to Gerolf Hoflehner from comment #46)
> @Jack: I can't reproduce your result. Could you double check? The block
> ordering issue I can reproduce and is consistent with the differences I see:
> the same instructions get generated, but the blocks are permuted. Thus some
> branch instructions and labsls could be different. I only looked at your
> ELFDumper.ii example. Perhaps there is a second problem shown by one of your
> other test cases. I used the ELFDumper.ii. Also, I used a stage 1 compiler
> in my (final) bisection. My build compiler is a stable release from last
> year.

The exact subset of object files whose compilation become non-deterministic
shifts the further back you go through the commits and becomes much less
frequent in occurrence (whereas current trunk produces the non-determinism
quite readily). My sense is that the issue may be almost completely latent near
the actual offending commit which may make it very difficult to find from a
traditional regression hunt.
Quuxplusone commented 7 years ago

One other observation is that the compilation of the ClangTidyDiagnosticConsumer.cpp, ELFDumper.cpp and SourceCoverageViewHTML.cpp source files in the stage2 bootstrap on x86_64 linux don't produce preprocessed source files which tigger the non-deterministic behavior (although those from the x86_64 darwin build still do under the x86_64 linux compiler). The two main differences in the compilation of those files in the x86_64 linux compiler build is that libstdc++ is used instead of libc++ on linux and darwin uses -fblocks.

Quuxplusone commented 7 years ago

The test cases still fail at r299813 so the non-determinism fixed in r299809 for the CoroSplit pass is unrelated.