run the libcxx tests

[LADSoft]

run the libcxx tests and add an appveyor build to run them as well,

[LADSoft]

@chuggafan, I was wondering if you had time and interest to try to make some sort of test harness for this? Makefiles or whatever you think necessary? I was thinking once we can run them reliably we could have a new appveyor configuration to run them in.

[chuggafan]

TBH I was mostly thinking in theory we can utilize CMake somehow and run the tests via their python... thing....

However I lack the knowledge of CMake to even BEGIN thinking of how to do this, @GitMensch would you have any idea?

[chuggafan]

Looking at it further, it turns out I have even less of an idea of where to begin after checking again than before, because the regular CMAKE stuff for libcxx is supposed to only be used in conjunction with a libcxxabi which would define the ABI portion of the libcxx interface that can change. Of course, we don't do this, but we also don't need to.

We'd probably have to do what microsoft in their STL did and write custom test harness code for the entire thing, while also marking known things that we don't support.

[GitMensch]

Where are the libcxx test and do we know how to execute them?

[chuggafan]

There's in the actual master repo for LLVM, as for executing them, you require a python script + cmake IIRC and none of that is what we have.

[chuggafan]

https://llvm.org/docs/CommandGuide/lit.html LIT is the tool they use for tests on LLVM AFAICT, MSVC uses this tool to test their implementation against the libcxx tests as well, if we can test our stuff against this it'd probably be good.

[LADSoft]

@chuggafan - i took a break from the optimizer problems to research this.

The LIT tool uses python but the libcxx tests don't use CMAKE... so I put #603 in milestone 4 rather than handle it now.

I made a LIT configuration file for orange c then made some minor tweaks to the libcxx 10 test runners so that it would at least configure with OCC and try to run tests. Right now it seems to be invoking the compiler but everything is failing quickly so I figure the test harness ​is passing command line parameters that OCC knows nothing about.

I have to get back to the optimization stuff tomorrow but if you are interested I will see if I can check in LIT and the libcxx test suite so that you can see it.

[chuggafan]

Check it in and tomorrow I'll check it out for sure.

[LADSoft]

ok thanks I appreciate it! I added the files, you should be able to go into \orangec\src and do

omake -flibcxxtests.mak

I put the tests in a folder near the library sources...

[LADSoft]

you can get python here if you don't already have it: https://www.python.org/

[chuggafan]

I see why the tests are failing, the way I dumped the commands being run is via adding a "--echo-all-commands" thing.

So one of the:

Command: ['c:/orangec/bin/occ.exe', '-o', 'C:\\Users\\User\\AppData\\Local\\Temp\\libcxx-testsuite-8wzj55tn\\test\\std\\containers\\unord\\unord.set\\Output\\swap_member.pass.cpp.o', '-x', 'c++', 'C:\\OrangeC\\src\\clibs\\cpp\\libcxx\\test\\std\\containers\\unord\\unord.set\\swap_member.pass.cpp', '-c', '-v', '-std=c++14', '-nostdinc++', '-I/orangec/src/clibs/cpp/libcxx\\include', '-I/orangec/src/clibs/cpp/libcxx\\test/support', '-Iinclude', '-Igenerated_headers', '-Itest', '-D_CRT_SECURE_NO_WARNINGS', '-DNOMINMAX', '-DLIBCXX_FILESYSTEM_STATIC_TEST_ROOT="C:\\OrangeC\\src\\clibs\\cpp\\libcxx\\test\\std\\input.output\\filesystems\\Inputs\\static_test_env"', '-DLIBCXX_FILESYSTEM_DYNAMIC_TEST_ROOT="C:\\Users\\User\\AppData\\Local\\Temp\\libcxx-testsuite-8wzj55tn\\test\\filesystem\\Output\\dynamic_env"', '-DLIBCXX_FILESYSTEM_DYNAMIC_TEST_HELPER="C:\\Users\\User\\AppData\\Local\\Programs\\Python\\Python39\\python.exe /orangec/src/clibs/cpp/libcxx\\test\\support\\filesystem_dynamic_test_helper.py"', '-c']

Of these, the invalid commands we have are:

-x c++, -std=c++14, -nostdinc++ if we implement all of these we should be able to do this.

For -std=c++14, we could just alias it to "you're good" and if we're passed anything else we can yell at the world saying "We don't support this version of the C++ standard".

For -x c++ this passes c++ to the linker nominally (I think? this should be double-checked elsewhere)

And -nostdinc++ disables the standard C++ include library, whilst -nostdinc does the same with C libraries, which is nice to have when you're testing something independently of your standard provided includes like we nominally are here...

Somehow 6 expected faileds passed btw, I dunno how but they did and I think this will get out a lot of the stuff we don't want...

After this we will have to tune the entire test system and see what is based off of clang behavior and that which isn't and work from there...

[GitMensch]

Sounds good. What about C (and other) standards specified that way?

[chuggafan]

Sounds good. What about C (and other) standards specified that way?

Sounds like #81 is now a blocker then ;)

[LADSoft]

well we are about to do c++17 so I wouldn't want to limit it to just c++14...

anyway occ doesn't really differentiate between the different C++ standards, the plan is just to support whatever the latest is. But I suppose at a minimum we could change the definition of __cplusplus.... and then let them use whatever features they want anyway...

[chuggafan]

The different versions of C++ have different deprecations so I'd rather we not claim to support a version of C++ we don't actually support... unless we're willing to do the "This is the different parsing standards" stuff.

[GitMensch]

I definitely like the idea of defining __cplusplus differnt as this allows people to compile their code that is conditional on that one. And if we don't want to limit something then the help text for -std= could say "changes some internal defines, but doesn't otherwise change diagnostic or compilation" and have the defines listed in the documentation.

[LADSoft]

yeah my main problem is there is a lot of work involved in figuring out the differences between all the versions... some of them are very subtle I think. And there is already a lot of work to be done... but we'd have to make it clear in the documentation we aren't enforcing standards compliance...

[LADSoft]

i went ahead and pushed a version with the command line changes, even though it won't build 100% properly.

[GitMensch]

Looks nice. Can you please copy+paste that for __STDC_VERSION__ definitions?

The value 199409L signifies the 1989 C standard as amended in 1994 [...]; the value 199901L signifies the 1999 revision of the C standard; the value 201112L signifies the 2011 revision of the C standard; the value 201710L signifies the 2017 revision of the C standard (which is otherwise identical to the 2011 version apart from correction of defects). An unspecified value larger [...] is used for the experimental [...] modes. This macro is not defined [...] when compiling C++ or Objective-C.

currently this value is hardcoded in https://github.com/LADSoft/OrangeC/blob/cdd3980ea71da2b0c03764e4ddd89e9ef2899bfc/src/occparse/osutil.cpp#L702 and https://github.com/LADSoft/OrangeC/blob/ca141d2ad3a82d307d369a01a8ae9f37a0f7ab4e/src/ocpp/ppMain.cpp#L166.

[LADSoft]

ok missed this, will get it addressed in the next push.

[chuggafan]

Did the test again, the issue now is (once I point the lit config directory to the right location):

********************
FAIL: libc++ :: std/diagnostics/syserr/syserr.compare/eq_error_code_error_code.pass.cpp (1463 of 5830)
******************** TEST 'libc++ :: std/diagnostics/syserr/syserr.compare/eq_error_code_error_code.pass.cpp' FAILED ********************
Command: ['c:/orangec/bin/occ.exe', '-o', 'C:\\Users\\User\\AppData\\Local\\Temp\\libcxx-testsuite-mu3b5ff9\\test\\std\\diagnostics\\syserr\\syserr.compare\\Output\\eq_error_code_error_code.pass.cpp.o', '-x', 'c++', 'C:\\OrangeC\\src\\clibs\\cpp\\libcxx\\test\\std\\diagnostics\\syserr\\syserr.compare\\eq_error_code_error_code.pass.cpp', '-c', '-v', '-Werror=thread-safety', '-std=c++14', '-nostdinc++', '-Ic:/orangec/src/clibs/cpp/libcxx\\include', '-Ic:/orangec/src/clibs/cpp/libcxx\\test/support', '-Iinclude', '-Igenerated_headers', '-Itest', '-D_CRT_SECURE_NO_WARNINGS', '-DNOMINMAX', '-DLIBCXX_FILESYSTEM_STATIC_TEST_ROOT="C:\\OrangeC\\src\\clibs\\cpp\\libcxx\\test\\std\\input.output\\filesystems\\Inputs\\static_test_env"', '-DLIBCXX_FILESYSTEM_DYNAMIC_TEST_ROOT="C:\\Users\\User\\AppData\\Local\\Temp\\libcxx-testsuite-mu3b5ff9\\test\\filesystem\\Output\\dynamic_env"', '-DLIBCXX_FILESYSTEM_DYNAMIC_TEST_HELPER="C:\\Users\\User\\AppData\\Local\\Programs\\Python\\Python39\\python.exe c:/orangec/src/clibs/cpp/libcxx\\test\\support\\filesystem_dynamic_test_helper.py"', '-c']
Exit Code: 1
Standard Error:
--
Fatal error: Cannot specify output file for multiple input files
occ Version 6.0.45.1 Copyright (C) LADSoft 2006-2020
--

Compilation failed unexpectedly!

Don't know what the issue is here since there seems to be only one file listed...

[chuggafan]

Also: we may need to modifiy some stuff such as test_macros.h to know if we support the C11 stuff in our libc...

[LADSoft]

probably something silly like the space between '-x' and 'c++' making c++ sound like a file. But I thought I tested that... I will look tonite...

[chuggafan]

The issue is actually the 2nd to last quoted item, might have to do with the TWO -c switches being used, as the filename list as read by the thing is:

Filename list:
File: C:\OrangeC\src\clibs\cpp\libcxx\test\std\input.output\file.streams\fstreams\ofstream.members\open_pointer.pass.cpp
File: C:/orangec/src/clibs/cpp/libcxx\test\support\filesystem_dynamic_test_helper.py
Filename list end

and if you take a look, this is commonly the 2nd to last item.

[LADSoft]

hm.... I took the command line from your previous post and fed it to occ, this didn't give a problem. But I was using a build of occ compiled with MSVC... it did enclude both -c's ... there is an issue though that the /Werror=xxx doesn't do what gnuc++ expects though and I will fix that soon.... anyway I'm waiting for a full build to see if there is something in the runtime's parsing of quotes at the moment. Beyond that I have no idea what it might be...

[LADSoft]

no luck there. But it seems like the quotes might be getting stripped off, I do get an error if I run it without them...

[chuggafan]

I dunno what's going on exactly other than the modification I did when there's an error saying there's multiple input files to print every file:

                    printf("Filename list:\n");
                    for (auto c = clist; c; c = c->next)
                    {
                        enter_filename((const char*)c->data);
                        printf("File: %s\n", (const char*)c->data);
                    }
                    printf("Filename list end\n");

Gives off what I listed above, which is why I drew my conclusion as I did.

The specific define is:

-DLIBCXX_FILESYSTEM_DYNAMIC_TEST_HELPER="C:\\Users\\User\\AppData\\Local\\Programs\\Python\\Python39\\python.exe c:/orangec/src/clibs/cpp/libcxx\\test\\support\\filesystem_dynamic_test_helper.py"

Which has the space inside of the quotations, which is what I drew my conclusion from, it might also be mismatched quotations or something else...

[LADSoft]

i'm unable to figure out how to get meaningful output from the tests, is there some place the results are stored so I could see what is going on? I'd like to echo the results of GetCommandLineA() to the console so I can see what is coming into occ, and diagnose a possible RTL issue... but don't know where to look for it... I guess I'm asking where you see the error?

[chuggafan]

I modified the makefile in the test directory to:

all:
    py \orangec\lit\lit.py --echo-all-commands std

And from there I just ctrl+c'd once stuff went wrong and copy/pasted the output, not sure about where results would be stored at all, maybe some temp directory? There's probably an option listed https://www.llvm.org/docs/CommandGuide/lit.html about it but I can't see one.

[LADSoft]

ok i just tried that a couple of different ways... I'm getting plenty of errors but not the error you showed me. The errors I'm getting are related to include_next not having a path (because of the -nostdinc++), and to the fact we still replace some of the libcxx headers with our own headers... did you do something to alleviate those?

[chuggafan]

Ah right, I forgot, the way I alieviated those was with at line 45 of lit.site.cfg:

lit_config.load_config(config, "c:/orangec/src/clibs/cpp/libcxx/test/lit.cfg") adding the C: there fixed it.

[LADSoft]

ok thanks that did it. Something is putting quotes around all the arguments... so arguments that already have quotes in them are double quoted and that confuses the quote processing, even in the MSVC runtime lib. not sure what to do about that yet... sounds like a topic for tomorrow though...

[LADSoft]

i fixed the quoting by changing the test harness... back to the problem with include_next now.

[chuggafan]

Maybe it has to do with the -nostdc++ include messing up include_next? I would hate for that to be the issue....

[LADSoft]

that was a good call. Turns out i didn't implement nostdc++ very well, just making sure it still had the C includes when erasing the C++ ones fixed that problem. Now there is some problem with constants... if you want you can see if it is a configuration error or I will look at it tomorrow nite.

[chuggafan]

I'll check out the fix tomorrow afternoon probably and add any findings here when I do.

[chuggafan]

There seems to be a lot of things referencing intrin.h from test_macros.h which we don't provide... there's also issues with for some reason ptrdiff_t is not being polled as the type when doing:

    typedef typename _VSTD::iterator_traits<_RandomAccessIterator>::difference_type difference_type;
    difference_type __len = __last - __first;

Not sure why this fails.

Also, some fails such as:

'c:/orangec/bin/occ.exe', '-o', 'C:\\Users\\User\\AppData\\Local\\Temp\\libcxx-testsuite-p15enm3i\\test\\std\\algorithms\\alg.sorting\\alg.heap.operations\\is.heap\\Output\\is_heap_until.pass.cpp.o', '-x', 'c++', 'C:\\OrangeC\\src\\clibs\\cpp\\libcxx\\test\\std\\algorithms\\alg.sorting\\alg.heap.operations\\is.heap\\is_heap_until.pass.cpp', '-c', '-v', '-Werror=thread-safety', '-std=c++14', '-nostdinc++', '-Ic:/orangec/src/clibs/cpp/libcxx\\include', '-Ic:/orangec/src/clibs/cpp/libcxx\\test/support', '-Iinclude', '-Igenerated_headers', '-Itest', '-D_CRT_SECURE_NO_WARNINGS', '-DNOMINMAX', '-DLIBCXX_FILESYSTEM_STATIC_TEST_ROOT="C:\\OrangeC\\src\\clibs\\cpp\\libcxx\\test\\std\\input.output\\filesystems\\Inputs\\static_test_env"', '-DLIBCXX_FILESYSTEM_DYNAMIC_TEST_ROOT="C:\\Users\\User\\AppData\\Local\\Temp\\libcxx-testsuite-p15enm3i\\test\\filesystem\\Output\\dynamic_env"', '-DLIBCXX_FILESYSTEM_DYNAMIC_TEST_HELPER="C:\\Users\\User\\AppData\\Local\\Programs\\Python\\Python39\\python.exe c:/orangec/src/clibs/cpp/libcxx\\test\\support\\filesystem_dynamic_test_helper.py"', '-c', '&&', 'c:/orangec/bin/occ.exe', '-o', 'C:\\Users\\User\\AppData\\Local\\Temp\\libcxx-testsuite-p15enm3i\\test\\std\\algorithms\\alg.sorting\\alg.heap.operations\\is.heap\\Output\\is_heap_until.pass.cpp.exe', 'C:\\Users\\User\\AppData\\Local\\Temp\\libcxx-testsuite-p15enm3i\\test\\std\\algorithms\\alg.sorting\\alg.heap.operations\\is.heap\\Output\\is_heap_until.pass.cpp.o', '-v']

The above generates:

Error c:\orangec\src\clibs\cpp\libcxx\test\support\test_macros.h(322): Could not open  <intrin.h>  for input
Error(232)    c:\orangec\src\clibs\cpp\libcxx\include\algorithm(4800):  Could not find a match for 'operator -(random_access_iterator<int*>, random_access_iterator<int*>)'
note:         c:\orangec\src\clibs\cpp\libcxx\include\algorithm(4798):  Referenced in instantiation of 'std::__1::is_heap_until<random_access_iterator<int*>, __less<int, int>>(random_access_iterator<int*>, random_access_iterator<int*>, __less<int, int>)'
note:         c:\orangec\src\clibs\cpp\libcxx\include\algorithm(4827):  Referenced in instantiation of 'std::__1::is_heap_until<random_access_iterator<int*>>(random_access_iterator<int*>, random_access_iterator<int*>)'
note:         C:\OrangeC\src\clibs\cpp\libcxx\test\std\algorithms\alg.sorting\alg.heap.operations\is.heap\is_heap_until.pass.cpp(37):  In template instantiation started here
2 Errors
Error: Undefined External 'main' in module c0xpe.nas
        1 Errors, 0 Warnings
        Errors encountered, not creating output file

Which... the main not being defined makes no sense

I also see a bunch of those "invalid constant" stuffs floating around.

Wow this will find a lot of problems.

[LADSoft]

this could be time consuming but it does need to be done... maybe we should concentrate on finishing up the atomics which could include at least cleaning up the libcxx tests for atomics (I haven't finished the optimizer work yet though), have a release, then immediately concentrate on this before going on to milestone 4? I'm saying that because it has been a long time since the last release and I don't know how long this will take... we've been delaying a release for a long time and it is overdue sigh.

[chuggafan]

If we get the atomics tests working I'd be happy with it, maybe we could label this milestone 3.5 as an intermediate step and then begin our work elsewhere?

Also the reason for the intrin.h stuff is that the libcxx team is using trickery to prevent a variable from being optimized away,

#if defined(__GNUC__) || defined(__clang__)
template <class Tp>
inline
void DoNotOptimize(Tp const& value) {
    asm volatile("" : : "r,m"(value) : "memory");
}

template <class Tp>
inline void DoNotOptimize(Tp& value) {
#if defined(__clang__)
  asm volatile("" : "+r,m"(value) : : "memory");
#else
  asm volatile("" : "+m,r"(value) : : "memory");
#endif
}
#else
#include <intrin.h>
template <class Tp>
inline void DoNotOptimize(Tp const& value) {
  const volatile void* volatile unused = __builtin_addressof(value);
  static_cast<void>(unused);
  _ReadWriteBarrier();
}
#endif

This is the code here that's causing it, I don't know enough about the builtins we have that would prevent a single variable from being optimized away at all...

[LADSoft]

ok that is cool, I can work with that. Just need to figure out the best way to handle this. Will probably have to look at where it is used to see what they are up to with it...

[LADSoft]

just for fun do you mind running the atomic tests to see how close they are to compiling?

[chuggafan]

Ah, I noticed that I do not actually have any idea how to do that automagically via suite, I'll make a command-line run-through of each test however and just have each one get run at the end of it...

[chuggafan]

Turns out it's having an issue with the .pass being found as an extension and therefore no .o being added and it's breaking things...

        occ -IC:\OrangeC\src\clibs\cpp\libcxx\test\support atomic_signal_fence.pass.cpp
occ Version 6.0.45.1 Copyright (C) LADSoft 2006-2020
Error: Input file 'atomic_signal_fence.o' does not exist.
Error: Undefined External 'main' in module c0xpe.nas
        2 Errors, 0 Warnings
        Errors encountered, not creating output file

Error makefile(11): commands returned error code 1

Wait, now that I think about it this might be an omake problem... YAAAAAAAAAAAAAAAAAAAAAAAY

[LADSoft]

cool :) IF you want to run them in batch, the makefile currently specifies 'std'. AFAIK this is the directory to look in for tests (it recurses). So you can just specify the relative path of whichever directory holds the atomic tests and let it go... it should find all the tests specifically related to atomics and run them.

[chuggafan]

You're right about the directory stuff, this is the current pass/fail list (once I basically elided the DoNotOptimize thing away):

PASS: libc++ :: std/atomics/atomics.flag/copy_volatile_assign.fail.cpp (1 of 57)
PASS: libc++ :: std/atomics/atomics.flag/copy_ctor.fail.cpp (2 of 57)
PASS: libc++ :: std/atomics/atomics.flag/copy_assign.fail.cpp (3 of 57)
PASS: libc++ :: std/atomics/atomics.flag/clear.pass.cpp (4 of 57)
UNSUPPORTED: libc++ :: std/atomics/atomics.lockfree/isalwayslockfree.pass.cpp (5 of 57)
PASS: libc++ :: std/atomics/atomics.fences/atomic_thread_fence.pass.cpp (6 of 57)
PASS: libc++ :: std/atomics/atomics.fences/atomic_signal_fence.pass.cpp (7 of 57)
PASS: libc++ :: std/atomics/atomics.flag/atomic_flag_clear.pass.cpp (8 of 57)
UNSUPPORTED: libc++ :: std/atomics/atomics.order/memory_order_new.pass.cpp (9 of 57)
FAIL: libc++ :: std/atomics/atomics.general/replace_failure_order.pass.cpp (10 of 57)
PASS: libc++ :: std/atomics/atomics.general/nothing_to_do.pass.cpp (11 of 57)
PASS: libc++ :: std/atomics/atomics.flag/init.pass.cpp (12 of 57)
FAIL: libc++ :: std/atomics/atomics.types.generic/bool.pass.cpp (13 of 57)
PASS: libc++ :: std/atomics/atomics.flag/atomic_flag_clear_explicit.pass.cpp (14 of 57)
PASS: libc++ :: std/atomics/atomics.lockfree/lockfree.pass.cpp (15 of 57)
PASS: libc++ :: std/atomics/atomics.types.generic/trivially_copyable.fail.cpp (16 of 57)
PASS: libc++ :: std/atomics/atomics.order/kill_dependency.pass.cpp (17 of 57)
PASS: libc++ :: std/atomics/atomics.syn/nothing_to_do.pass.cpp (18 of 57)
FAIL: libc++ :: std/atomics/atomics.types.generic/integral.pass.cpp (19 of 57)
PASS: libc++ :: std/atomics/atomics.order/memory_order.pass.cpp (20 of 57)
PASS: libc++ :: std/atomics/atomics.types.generic/cstdint_typedefs.pass.cpp (21 of 57)
FAIL: libc++ :: std/atomics/atomics.types.generic/address.pass.cpp (22 of 57)
FAIL: libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_compare_exchange_strong.pass.cpp (23 of 57)
FAIL: libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_compare_exchange_strong_explicit.pass.cpp (24 of 57)
PASS: libc++ :: std/atomics/atomics.types.generic/integral_typedefs.pass.cpp (25 of 57)
PASS: libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.arith/nothing_to_do.pass.cpp (26 of 57)
FAIL: libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_compare_exchange_weak.pass.cpp (27 of 57)
FAIL: libc++ :: std/atomics/atomics.types.generic/trivially_copyable.pass.cpp (28 of 57)
FAIL: libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_compare_exchange_weak_explicit.pass.cpp (29 of 57)
FAIL: libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_exchange.pass.cpp (30 of 57)
FAIL: libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_exchange_explicit.pass.cpp (31 of 57)
PASS: libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.pointer/nothing_to_do.pass.cpp (32 of 57)
FAIL: libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_fetch_add.pass.cpp (33 of 57)
FAIL: libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_fetch_add_explicit.pass.cpp (34 of 57)
PASS: libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.general/nothing_to_do.pass.cpp (35 of 57)
FAIL: libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_fetch_and.pass.cpp (36 of 57)
FAIL: libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_fetch_and_explicit.pass.cpp (37 of 57)
FAIL: libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_fetch_or.pass.cpp (38 of 57)
FAIL: libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_fetch_or_explicit.pass.cpp (39 of 57)
FAIL: libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_fetch_sub.pass.cpp (40 of 57)
FAIL: libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_fetch_sub_explicit.pass.cpp (41 of 57)
FAIL: libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_fetch_xor.pass.cpp (42 of 57)
FAIL: libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_fetch_xor_explicit.pass.cpp (43 of 57)
FAIL: libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_init.pass.cpp (44 of 57)
FAIL: libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_is_lock_free.pass.cpp (45 of 57)
FAIL: libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_load.pass.cpp (46 of 57)
FAIL: libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_load_explicit.pass.cpp (47 of 57)
FAIL: libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_store.pass.cpp (48 of 57)
FAIL: libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_store_explicit.pass.cpp (49 of 57)
FAIL: libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/ctor.pass.cpp (50 of 57)
FAIL: libc++ :: std/atomics/atomics.flag/test_and_set.pass.cpp (51 of 57)
PASS: libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.templ/nothing_to_do.pass.cpp (52 of 57)
PASS: libc++ :: std/atomics/atomics.types.operations/nothing_to_do.pass.cpp (53 of 57)
FAIL: libc++ :: std/atomics/atomics.flag/atomic_flag_test_and_set.pass.cpp (54 of 57)
FAIL: libc++ :: std/atomics/atomics.flag/atomic_flag_test_and_set_explicit.pass.cpp (55 of 57)
PASS: libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_var_init.pass.cpp (56 of 57)
FAIL: libc++ :: std/atomics/atomics.flag/default.pass.cpp (57 of 57)

So this is the list of stuff we need to work on: (edited in the list of failing tests)

    libc++ :: std/atomics/atomics.flag/atomic_flag_test_and_set.pass.cpp
    libc++ :: std/atomics/atomics.flag/atomic_flag_test_and_set_explicit.pass.cpp
    libc++ :: std/atomics/atomics.flag/default.pass.cpp
    libc++ :: std/atomics/atomics.flag/test_and_set.pass.cpp
    libc++ :: std/atomics/atomics.general/replace_failure_order.pass.cpp
    libc++ :: std/atomics/atomics.types.generic/address.pass.cpp
    libc++ :: std/atomics/atomics.types.generic/bool.pass.cpp
    libc++ :: std/atomics/atomics.types.generic/integral.pass.cpp
    libc++ :: std/atomics/atomics.types.generic/trivially_copyable.pass.cpp
    libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_compare_exchange_strong.pass.cpp
    libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_compare_exchange_strong_explicit.pass.cpp
    libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_compare_exchange_weak.pass.cpp
    libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_compare_exchange_weak_explicit.pass.cpp
    libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_exchange.pass.cpp
    libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_exchange_explicit.pass.cpp
    libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_fetch_add.pass.cpp
    libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_fetch_add_explicit.pass.cpp
    libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_fetch_and.pass.cpp
    libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_fetch_and_explicit.pass.cpp
    libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_fetch_or.pass.cpp
    libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_fetch_or_explicit.pass.cpp
    libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_fetch_sub.pass.cpp
    libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_fetch_sub_explicit.pass.cpp
    libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_fetch_xor.pass.cpp
    libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_fetch_xor_explicit.pass.cpp
    libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_init.pass.cpp
    libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_is_lock_free.pass.cpp
    libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_load.pass.cpp
    libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_load_explicit.pass.cpp
    libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_store.pass.cpp
    libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/atomic_store_explicit.pass.cpp
    libc++ :: std/atomics/atomics.types.operations/atomics.types.operations.req/ctor.pass.cpp

[LADSoft]

some of that may be because the optimizers are getting in the way. I haven't fully supported atomics especially in the gcse optimizer... Be interesting to see if there is a difference when using /O- on the command line... but I don't know how to configure the test harness to do that.

I did figure out that you are somewhat right about the .o files... they are being produced but without a .o extension. Similarly for .s files. The .pass thing is getting in the way... I will have a fix tonite...

Anyway I'll take a break from the optimizer work and look at this tonite.

[chuggafan]

There seems to be no difference with /O- and not, I added it in and checked but it's the same 32 passes failing afaict.

[LADSoft]

ok thanks :smile:

[chuggafan]

Doing some testing on the other sections, such as the numerics, it seems that libcxx expects their versions of headers for the C overrides to appear if -nostdinc++ is done... which might mean another layer of include shenanigans if we want to support this (e.g. we do C++ headers -> C++ version of C headers -> C headers), which is also near-impossible to separate.

I will eventually look at how clang differentiates this...

[chuggafan]

I'm now thoroughly confused after doing an investigation with godbolt, showing that it doesn't do what I thought it did and this doesn't happen at all.... it might legit be expecting that _Complex is supported in C++ mode... I will look into whether this is standards compliant....