comparing clang vs. gcc compiled regtest results [x86, clang]

[nbriggs]

This is more informational than a report of a specific problem. Using valgrind (valgrind-3.17.0.GIT-39c9c1ec89-20200621X) compiled with clang 9.0.1 running the regtest compiled either with the same clang, or with gcc ((FreeBSD Ports Collection) 9.3.0) shows up the following differences in test failures:

clang0620.txt gcc0620.txt

$ diff /tmp/gcc0620.txt /tmp/clang0620.txt
1c1,2
< == 646 tests, 19 stderr failures, 2 stdout failures, 2 stderrB failures, 3 stdoutB failures, 0 post failures ==
---
> == 646 tests, 34 stderr failures, 2 stdout failures, 3 stderrB failures, 3 stdoutB failures, 0 post failures ==
> gdbserver_tests/mcblocklistsearch        (stderrB)
11a13
> memcheck/tests/fprw                      (stderr)
12a15,21
> memcheck/tests/leak-cases-full           (stderr)
> memcheck/tests/leak-cases-summary        (stderr)
> memcheck/tests/leak-cycle                (stderr)
> memcheck/tests/leak-tree                 (stderr)
> memcheck/tests/leak_cpp_interior         (stderr)
> memcheck/tests/lks                       (stderr)
> memcheck/tests/origin5-bz2               (stderr)
14a24,25
> memcheck/tests/varinfo5                  (stderr)
> memcheck/tests/varinfo6                  (stderr)
18a30,31
> drd/tests/annotate_trace_memory          (stderr)
> drd/tests/annotate_trace_memory_xml      (stderr)
21a35,37
> drd/tests/omp_matinv                     (stderr)
> drd/tests/omp_matinv_racy                (stderr)
> drd/tests/omp_prime_racy                 (stderr)
$ 

[paulfloyd]

For the varinfo/origins and leaks/gdbserver testcases, at least we have some idea of what is going wrong.

fprw just looks like a codegen difference causing different types of error.

I haven't analyzed the others. annotate_trace_memory has a difference in the order of the traces.

The OpenMP testcase failures are either because we're missing some suppressions or else there is a missing libomp redirection.

For comparison on amd64 I get the following extra failures

< memcheck/tests/amd64/insn-pmovmskb (stderr) < memcheck/tests/clientperm (stderr) < memcheck/tests/signal2 (stdout) < memcheck/tests/signal2 (stderr) < helgrind/tests/tc22_exit_w_lock (stderr) < drd/tests/atomic_var (stderr) < drd/tests/tc04_free_lock (stderr) < none/tests/amd64/ssse3_misaligned (stderr)

But these one pass on amd64

gdbserver_tests/mcblocklistsearch memcheck/tests/leak-tree memcheck/tests/leak_cpp_interior memcheck/tests/varinfo5 drd/tests/annotate_trace_memory drd/tests/annotate_trace_memory_xml

[paulfloyd]

I've moved this info to the wiki page.