foss-for-synopsys-dwc-arc-processors / arc-gnu-toolchain

Scripting for building ARC toolchain
5 stars 3 forks source link

Investigate failure rate difference when using a dg board with a wrapper or wihout. #19

Closed claziss closed 1 year ago

claziss commented 1 year ago

Using the default arc-sim.exp board without wrapper, we have a number of extra failures. Those are:

           "FAIL: gcc.dg/pr82274-1.c execution test",
           "XPASS: gcc.dg/tree-ssa/20040204-1.c scan-tree-dump-times optimized \"link_error\" 0",
           "XPASS: c-c++-common/raw-string-7.c  -Wc++-compat  execution test"
           "XPASS: g++.dg/tls/thread_local-order2.C"
           "FAIL: gcc.dg/tree-prof/pr79587.c execution"
           "FAIL: gcc.dg/torture/fp-int-convert-float16-timode.c"

With the wrapper added (i.e., set_board_info needs_status_wrapper 1), we get:

PASS: gcc.dg/pr82274-1.c execution test
XFAIL: c-c++-common/raw-string-7.c  -Wc++-compat  execution test
XFAIL: g++.dg/tls/thread_local-order2.C  -std=c++20 execution test
PASS: gcc.dg/tree-prof/pr79587.c execution
UNSUPPORTED: gcc.dg/torture/fp-int-convert-float16-timode.c 
luismgsilva commented 1 year ago

The Newlib testsuite filter allowlist has been updated.

All the displayed tests are exhibiting the expected behavior. All tests were executed in debug mode to ensure that the output and behavior of the tests are accurate.

The following list contains all the tests that currently produce different outputs depending on whether the wrapper is enabled or disabled. This discrepancy might be related to how Newlib handles the abort process, as all of these tests exhibit the expected behavior of aborting.

pr82274-1.c

TEST NON-WRAPPER WRAPPER 1
pr82274-1c FAIL PASS

Test Case

/* PR target/82274 */
/* { dg-do run } */
/* { dg-shouldfail "trapv" } */
/* { dg-options "-ftrapv" } */

#include <stdlib.h>

int
main ()
{
  abort();

#ifdef __SIZEOF_INT128__
  volatile __int128 m = -(((__int128) 1) << (__CHAR_BIT__ * __SIZEOF_INT128__ / 2));
#else
  volatile long long m = -(1LL << (__CHAR_BIT__ * __SIZEOF_LONG_LONG__ / 2));
#endif
  m = m * m;
  return 0;
}

This test case tests the behavior of the -ftrapv flag when integer overflow occurs during a multiplication operation. Is designed to verify that GCC correctly detects integer overflow and aborts the program as expected when the ftrapv flag is enabled. When executing the test, GCC should correctly detect the integer overflow caused by the multiplication and abort the program as expected, resulting in a test failure. If the -ftrapv flag is not enabled, the program will complete without error, resulting in a test failure because of the dg-shouldfail dejagnu directive.

thread_local-order2.C

TEST NON-WRAPPER WRAPPER 1
thread_local-order2.C XPASS XFAIL

Test Case

// The standard says that a1 should be destroyed before a0 even though
// that isn't reverse order of construction.  We need to move
// __cxa_thread_atexit into glibc to get this right.

// { dg-do run }
// { dg-require-effective-target c++11 }
// { dg-add-options tls }
// { dg-require-effective-target tls_runtime }
// { dg-xfail-run-if "" { { hppa*-*-hpux* *-*-solaris* } || { newlib } } }

extern "C" void abort();
extern "C" int printf (const char *, ...);
#define printf(...)

int c;
struct A {
  int i;
  A(int i): i(i) { printf ("A(%d)\n", i); ++c; }
  ~A() { printf("~A(%d)\n", i); if (i != --c) abort(); }
};

thread_local A a1(1);
A* ap = &a1;
A a0(0);

int main()
{
  if (c != 2) abort();
}

The test initially returns *** EXIT code 0 from the main function, indicating a successful completion without any issues. However, after the test completes, the destructors of the struct instances are called. Due to the lack of Thread-Local Storage (TLS) support in the baremetal environment, the destructors are executed in a linear order. This deviation from the expected order causes the test to return an abort with the code *** EXIT code 4242, as anticipated.

raw-string-7.c

TEST NON-WRAPPER WRAPPER 1
raw-string-7.c XPASS XFAIL

Test Case

// The trailing whitespace after \ and before newline extension
// breaks full compliance for raw strings.
// { dg-do run { xfail *-*-* } }
// { dg-options "-std=gnu99" { target c } }
// { dg-options "-std=c++0x" { target c++ } }

// Note, there is a single space after \ on the following line.
const char *s0 = R"(\ 
)";
// { dg-bogus "backslash and newline separated by space" "" { xfail *-*-* } 8 }

// Note, there is a single tab after \ on the following line.
const char *s1 = R"(\   
)";
// { dg-bogus "backslash and newline separated by space" "" { xfail *-*-* } 13 }

int
main (void)
{
  if (__builtin_strcmp (s0, "\\ \n") != 0
      || __builtin_strcmp (s1, "\\\t\n") != 0)
    __builtin_abort ();
  return 0;
}

The test case checks if the raw string literals are being properly handled by the compiler by defining two string constants s0 and s1. s0 contains a backslash followed by a space and a newline, while s1 contains a backslash followed by a tab and a newline. The test case expects that the compiler will treat these are raw string literals and remove the escape characters from the string constrants. The __builtin_strcmp functions is used to compare the resulting string constants with the expected values of "\n" and "\t\n". If the string constants do not match the expected values, the test case will fail and trigger an __builtin_abort() function call. The xfail directive indicates that the test is expected to fail on all platforms.