search

Windows-on-ARM-Experiments / mingw-woarm64-build

Workflows and build scripts for Windows on Arm64 GNU cross-compiler for `aarch64-w64-mingw32` target.
GNU General Public License v2.0
32 stars 4 forks source link

Add script for grouping test results by source files #184

Closed vejbomar closed 2 months ago

vejbomar commented 2 months ago

The script helps to identify what areas have potential to increase the test coverage most and thus it might be a good idea to prioritize patching them.

vejbomar commented 2 months ago

Self review: I thought that tests write their output and then a line with the result keyword like PASS: ... or FAIL: ..., ... After looking more closely into the tests, I realized this is not really true. There are usually multiple lines belonging to several test = lines with result keyword and even there can be some output after them.

I think there are two options now.

1) I can change the script, so that it correctly parses lines and their corresponding results. Currently, it's overkill for what it does but it would be prepared for adding more complex grouping options (e.g. analyzing messages and grouping semantically by the same errors).

2) Change the script, so it reads only *.sum files which contains only one line for each test result. This allows to significantly simplify the script. This would IMO make sense if we won't add more functionality into the script = it would be used solely for grouping by source filepaths like now.

Opinions @eukarpov @Blackhex ?

Example of logs that are relevant to tests in one file (https://github.com/Windows-on-ARM-Experiments/gcc-woarm64/blob/woarm64/gcc/testsuite/c-c%2B%2B-common/analyzer/realloc-1.c):

Executing on host: /root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/gcc/testsuite/g++/../../xg++ -B/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/gcc/testsuite/g++/../../ /root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c  -fdiagnostics-plain-output  -nostdinc++ -I/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/aarch64-w64-mingw32/libstdc++-v3/include/aarch64-w64-mingw32 -I/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/aarch64-w64-mingw32/libstdc++-v3/include -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/libsupc++ -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/include/backward -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/testsuite/util -fmessage-length=0  -std=c++17 -fanalyzer -Wanalyzer-too-complex -fanalyzer-call-summaries -Wno-free-nonheap-object  -S -o realloc-1.s    (timeout = 5000)
spawn -ignore SIGHUP /root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/gcc/testsuite/g++/../../xg++ -B/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/gcc/testsuite/g++/../../ /root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c -fdiagnostics-plain-output -nostdinc++ -I/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/aarch64-w64-mingw32/libstdc++-v3/include/aarch64-w64-mingw32 -I/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/aarch64-w64-mingw32/libstdc++-v3/include -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/libsupc++ -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/include/backward -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/testsuite/util -fmessage-length=0 -std=c++17 -fanalyzer -Wanalyzer-too-complex -fanalyzer-call-summaries -Wno-free-nonheap-object -S -o realloc-1.s
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c: In function 'void test_10(char*, int)':
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c:94:20: warning: ignoring return value of 'void* __builtin_realloc(void*, long long unsigned int)' declared with attribute 'warn_unused_result' [-Wunused-result]
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c: In function 'void test_2(void*)':
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c:28:1: warning: leak of 'p' [CWE-401] [-Wanalyzer-malloc-leak]
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c:25:20: note: (1) allocated here
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c:26:15: note: (2) when 'realloc' fails
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c:28:1: note: (3) 'p' leaks here; was allocated at (1)
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c: In function 'int* test_5(int*)':
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c:49:6: warning: dereference of NULL 'q' [CWE-476] [-Wanalyzer-null-dereference]
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c:48:28: note: (1) when 'realloc' fails
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c:50:10: note: (2) 'q' is NULL
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c:49:6: note: (3) dereference of NULL 'q'
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c: In function 'void test_6(size_t)':
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c:56:1: warning: leak of 'p' [CWE-401] [-Wanalyzer-malloc-leak]
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c:55:21: note: (1) allocated here
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c:55:21: note: (2) when 'realloc' succeeds, moving buffer
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c:56:1: note: (3) 'p' leaks here; was allocated at (1)
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c: In function 'void* test_7(size_t)':
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c:63:21: warning: 'realloc' of '& buf' which points to memory on the stack [CWE-590] [-Wanalyzer-free-of-non-heap]
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c:62:8: note: (1) region created on stack here
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c:62:8: note: (2) capacity: 100 bytes
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c:63:21: note: (3) call to 'realloc' here
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c: In function 'void test_8()':
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c:81:21: warning: 'p' should have been deallocated with 'foo_release' but was deallocated with 'realloc' [CWE-762] [-Wanalyzer-mismatching-deallocation]
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c:80:31: note: (1) allocated here (expects deallocation with 'foo_release')
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c:81:21: note: (2) deallocated with 'realloc' here; allocation at (1) expects deallocation with 'foo_release'
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c: In function 'void test_9(void*)':
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c:89:21: warning: double-'free' of 'p' [CWE-415] [-Wanalyzer-double-free]
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c:88:8: note: (1) first 'free' here
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c:89:21: note: (2) second 'free' here; first 'free' was at (1)
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c: In function 'void test_10(char*, int)':
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c:96:1: warning: leak of '<unknown>' [CWE-401] [-Wanalyzer-malloc-leak]
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c:96:1: note: (1) '<unknown>' leaks here
PASS: c-c++-common/analyzer/realloc-1.c  -std=c++17  (test for warnings, line 25)
PASS: c-c++-common/analyzer/realloc-1.c  -std=c++17  (test for warnings, line 26)
PASS: c-c++-common/analyzer/realloc-1.c  -std=c++17  (test for warnings, line 28)
PASS: c-c++-common/analyzer/realloc-1.c  -std=c++17  (test for warnings, line 48)
PASS: c-c++-common/analyzer/realloc-1.c  -std=c++17  (test for warnings, line 49)
PASS: c-c++-common/analyzer/realloc-1.c  -std=c++17  (test for warnings, line 56)
PASS: c-c++-common/analyzer/realloc-1.c  -std=c++17  (test for warnings, line 62)
PASS: c-c++-common/analyzer/realloc-1.c  -std=c++17  (test for warnings, line 63)
PASS: c-c++-common/analyzer/realloc-1.c  -std=c++17  (test for warnings, line 81)
PASS: c-c++-common/analyzer/realloc-1.c  -std=c++17  (test for warnings, line 89)
FAIL: c-c++-common/analyzer/realloc-1.c  -std=c++17  at line 95 (test for warnings, line 94)
PASS: c-c++-common/analyzer/realloc-1.c  -std=c++17  (test for warnings, line 96)
FAIL: c-c++-common/analyzer/realloc-1.c  -std=c++17 (test for excess errors)
Excess errors:
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/c-c++-common/analyzer/realloc-1.c:94:20: warning: ignoring return value of 'void* __builtin_realloc(void*, long long unsigned int)' declared with attribute 'warn_unused_result' [-Wunused-result]
Blackhex commented 2 months ago

Opinions @eukarpov @Blackhex ?

If we would not consider extending this script, getting the current statistics can be done on *.sum files only. On the other hand, IMO the extension of this script is expected and for now it's completely fine that it ignores the Excess errors.

vejbomar commented 2 months ago

Opinions @eukarpov @Blackhex ?

If we would not consider extending this script, getting the current statistics can be done on *.sum files only. On the other hand, IMO the extension of this script is expected and for now it's completely fine that it ignores the Excess errors.

I think the current state is not good. Not only Excess errors are parsed wrongly, it also parses all the log lines without result token together with the first line containing result token. So IMO it would be unexpected that messages are loaded incorrectly if someone else would want to do changes in the script in the future.

Illustration what I mean. These log messages:

Executing on host: /root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/gcc/testsuite/g++/../../xg++ -B/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/gcc/testsuite/g++/../../ /root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/g++.dg/parse/unnamed1.C  -fdiagnostics-plain-output  -nostdinc++ -I/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/aarch64-w64-mingw32/libstdc++-v3/include/aarch64-w64-mingw32 -I/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/aarch64-w64-mingw32/libstdc++-v3/include -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/libsupc++ -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/include/backward -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/testsuite/util -fmessage-length=0  -std=c++26  -pedantic-errors -Wno-long-long  -S -o unnamed1.s    (timeout = 5000)
spawn -ignore SIGHUP /root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/gcc/testsuite/g++/../../xg++ -B/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/gcc/testsuite/g++/../../ /root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/g++.dg/parse/unnamed1.C -fdiagnostics-plain-output -nostdinc++ -I/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/aarch64-w64-mingw32/libstdc++-v3/include/aarch64-w64-mingw32 -I/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/aarch64-w64-mingw32/libstdc++-v3/include -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/libsupc++ -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/include/backward -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/testsuite/util -fmessage-length=0 -std=c++26 -pedantic-errors -Wno-long-long -S -o unnamed1.s
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/g++.dg/parse/unnamed1.C:5:14: error: static data member '<unnamed struct>::i' in unnamed class [-fpermissive]
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/g++.dg/parse/unnamed1.C:4:16: note: unnamed class defined here
compiler exited with status 1
PASS: g++.dg/parse/unnamed1.C  -std=c++26  (test for warnings, line 4)
PASS: g++.dg/parse/unnamed1.C  -std=c++26  (test for errors, line 5)
PASS: g++.dg/parse/unnamed1.C  -std=c++26 (test for excess errors)

are currently parsed as 3 messages:

Executing on host: /root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/gcc/testsuite/g++/../../xg++ -B/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/gcc/testsuite/g++/../../ /root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/g++.dg/parse/unnamed1.C  -fdiagnostics-plain-output  -nostdinc++ -I/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/aarch64-w64-mingw32/libstdc++-v3/include/aarch64-w64-mingw32 -I/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/aarch64-w64-mingw32/libstdc++-v3/include -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/libsupc++ -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/include/backward -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/testsuite/util -fmessage-length=0  -std=c++26  -pedantic-errors -Wno-long-long  -S -o unnamed1.s    (timeout = 5000)
spawn -ignore SIGHUP /root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/gcc/testsuite/g++/../../xg++ -B/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/gcc/testsuite/g++/../../ /root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/g++.dg/parse/unnamed1.C -fdiagnostics-plain-output -nostdinc++ -I/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/aarch64-w64-mingw32/libstdc++-v3/include/aarch64-w64-mingw32 -I/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/aarch64-w64-mingw32/libstdc++-v3/include -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/libsupc++ -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/include/backward -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/testsuite/util -fmessage-length=0 -std=c++26 -pedantic-errors -Wno-long-long -S -o unnamed1.s
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/g++.dg/parse/unnamed1.C:5:14: error: static data member '<unnamed struct>::i' in unnamed class [-fpermissive]
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/g++.dg/parse/unnamed1.C:4:16: note: unnamed class defined here
compiler exited with status 1
PASS: g++.dg/parse/unnamed1.C  -std=c++26  (test for warnings, line 4)
PASS: g++.dg/parse/unnamed1.C  -std=c++26  (test for errors, line 5)
PASS: g++.dg/parse/unnamed1.C  -std=c++26 (test for excess errors)

Instead it should rather be parsed as those 3 messages, since all the lines are relevant to each of the test result:

Executing on host: /root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/gcc/testsuite/g++/../../xg++ -B/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/gcc/testsuite/g++/../../ /root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/g++.dg/parse/unnamed1.C  -fdiagnostics-plain-output  -nostdinc++ -I/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/aarch64-w64-mingw32/libstdc++-v3/include/aarch64-w64-mingw32 -I/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/aarch64-w64-mingw32/libstdc++-v3/include -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/libsupc++ -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/include/backward -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/testsuite/util -fmessage-length=0  -std=c++26  -pedantic-errors -Wno-long-long  -S -o unnamed1.s    (timeout = 5000)
spawn -ignore SIGHUP /root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/gcc/testsuite/g++/../../xg++ -B/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/gcc/testsuite/g++/../../ /root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/g++.dg/parse/unnamed1.C -fdiagnostics-plain-output -nostdinc++ -I/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/aarch64-w64-mingw32/libstdc++-v3/include/aarch64-w64-mingw32 -I/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/aarch64-w64-mingw32/libstdc++-v3/include -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/libsupc++ -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/include/backward -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/testsuite/util -fmessage-length=0 -std=c++26 -pedantic-errors -Wno-long-long -S -o unnamed1.s
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/g++.dg/parse/unnamed1.C:5:14: error: static data member '<unnamed struct>::i' in unnamed class [-fpermissive]
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/g++.dg/parse/unnamed1.C:4:16: note: unnamed class defined here
compiler exited with status 1
PASS: g++.dg/parse/unnamed1.C  -std=c++26  (test for warnings, line 4)
Executing on host: /root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/gcc/testsuite/g++/../../xg++ -B/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/gcc/testsuite/g++/../../ /root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/g++.dg/parse/unnamed1.C  -fdiagnostics-plain-output  -nostdinc++ -I/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/aarch64-w64-mingw32/libstdc++-v3/include/aarch64-w64-mingw32 -I/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/aarch64-w64-mingw32/libstdc++-v3/include -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/libsupc++ -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/include/backward -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/testsuite/util -fmessage-length=0  -std=c++26  -pedantic-errors -Wno-long-long  -S -o unnamed1.s    (timeout = 5000)
spawn -ignore SIGHUP /root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/gcc/testsuite/g++/../../xg++ -B/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/gcc/testsuite/g++/../../ /root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/g++.dg/parse/unnamed1.C -fdiagnostics-plain-output -nostdinc++ -I/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/aarch64-w64-mingw32/libstdc++-v3/include/aarch64-w64-mingw32 -I/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/aarch64-w64-mingw32/libstdc++-v3/include -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/libsupc++ -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/include/backward -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/testsuite/util -fmessage-length=0 -std=c++26 -pedantic-errors -Wno-long-long -S -o unnamed1.s
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/g++.dg/parse/unnamed1.C:5:14: error: static data member '<unnamed struct>::i' in unnamed class [-fpermissive]
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/g++.dg/parse/unnamed1.C:4:16: note: unnamed class defined here
compiler exited with status 1
PASS: g++.dg/parse/unnamed1.C  -std=c++26  (test for errors, line 5)
Executing on host: /root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/gcc/testsuite/g++/../../xg++ -B/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/gcc/testsuite/g++/../../ /root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/g++.dg/parse/unnamed1.C  -fdiagnostics-plain-output  -nostdinc++ -I/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/aarch64-w64-mingw32/libstdc++-v3/include/aarch64-w64-mingw32 -I/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/aarch64-w64-mingw32/libstdc++-v3/include -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/libsupc++ -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/include/backward -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/testsuite/util -fmessage-length=0  -std=c++26  -pedantic-errors -Wno-long-long  -S -o unnamed1.s    (timeout = 5000)
spawn -ignore SIGHUP /root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/gcc/testsuite/g++/../../xg++ -B/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/gcc/testsuite/g++/../../ /root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/g++.dg/parse/unnamed1.C -fdiagnostics-plain-output -nostdinc++ -I/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/aarch64-w64-mingw32/libstdc++-v3/include/aarch64-w64-mingw32 -I/root/mingw-woarm64-build/build-aarch64-w64-mingw32-msvcrt/gcc/aarch64-w64-mingw32/libstdc++-v3/include -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/libsupc++ -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/include/backward -I/root/mingw-woarm64-build/code/gcc-master/libstdc++-v3/testsuite/util -fmessage-length=0 -std=c++26 -pedantic-errors -Wno-long-long -S -o unnamed1.s
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/g++.dg/parse/unnamed1.C:5:14: error: static data member '<unnamed struct>::i' in unnamed class [-fpermissive]
/root/mingw-woarm64-build/code/gcc-master/gcc/testsuite/g++.dg/parse/unnamed1.C:4:16: note: unnamed class defined here
compiler exited with status 1
PASS: g++.dg/parse/unnamed1.C  -std=c++26 (test for excess errors)
eukarpov commented 2 months ago

Thanks for the observation about the parsing issue. I think it is ok for now because we group results by path. Yes, it is not accurate, however it helps to build common picture. Adding tokens to every line should help to process result correctly.

vejbomar commented 2 months ago

Updated code based on the last feedback and also started new pipeline to be sure everything works after changes: https://github.com/Windows-on-ARM-Experiments/mingw-woarm64-build/actions/runs/10598941700

vejbomar commented 2 months ago

@Blackhex can you take a look at the last commit here? I added the ROOT_DIR to config.sh and refactored one other script to use it there. So I don't want to sneak in that change after approval.