run the libcxx tests

[LADSoft]

run the libcxx tests and add an appveyor build to run them as well,

[chuggafan]

Reading through, this seems to not be standards compliant in total, _Complex is not mentioned at all and support for it is supposed to be kicked away. Clang seems to include support for it just for -nostdinc++ for some reason, as does GCC. Dunno why, but it ain't standard... MSVC doesn't if only because they don't support getting rid of the stdinc headers.

[LADSoft]

yeah i don't know what to say about _Complex. My understanding was it isn't supported in C++ so I've never tried to support it... I suppose in a pinch we might be able to get it working though...

[chuggafan]

We could just munch it and then turn around and give warnings as an extension just for this scenario....

[chuggafan]

While testing for some stuff I have determined that the macros limiting C++ access to stdint.h are all noncompliant with the C++ standard.

1 The header also defines numerous macros of the form: INT_[FAST LEAST]{8 16 32 64}MIN [U]INT[FAST LEAST]{8 16 32 64}_MAX INT{MAX PTR}_MIN [U]INT{MAX PTR}_MAX {PTRDIFF SIG_ATOMIC WCHAR WINT}{_MAX _MIN} SIZE_MAX plus function macros of the form: [U]INT{8 16 32 64 MAX}_C 2 The header defines all types and macros the same as the C standard library header . See also: ISO C 7.20

I'll make a PR at some other point if you don't want to remove the 2 ifdefs.

[chuggafan]

I've noted while running the tests language.support/support.limits oftentimes the compiler will exit with 0xC0000005, meaning a big issue is going on with those...

[chuggafan]

The header platform_support.h doesn't work because we don't either have:

GetTempPath and GetTempFileName in io.h (along with MAX_PATH) or _mktemp_s support.

I vote for the former rather than the latter, but the latter can be done if wanted...

[chuggafan]

https://github.com/microsoft/STL/blob/e745bad3b1d05b5b19ec652d68abb37865ffa454/tests/libcxx/expected_results.txt

I'm just leaving this here because this explains a lot of what seems to be wrong with the current libcxx tests and what Microsoft's STL is doing to get around it

When doing investigations this should be referenced at some points in case there's failures that we are encountering: If we need to exactly match the libcxx behavior so be it but if it already passes the standard behavior then we're good.

[LADSoft]

thanks for looking into this, will make the work just a little easier.

[chuggafan]

BTW RE: the default constructor thing, I think tomorrow I'll send off an email to the WG asking for a definitive answer, I'm assuming this should be the case considering libcxx can be used with GCC or CLANG and MSVC seems to get around this by just adding a {} along with defaulting the constructor, meaning the type gets initialized too... as for why 0? I think it's cuz it's empty brace.

[LADSoft]

i think if you specify an empty bracket initializer for the member declaration, that would mean use the default constructor for a scalar, which is to set it to zero. But the libcxx sources didn't do that either...

I guess at issue is whether or not default constructing a structure default constructs all scalar/pointer members of that structure as well as default constructing all contained structures and base classes... as far as I know it doesn't but I'm open to being wrong :smile:

[chuggafan]

Now that I think about it, I think the requirement for the default constructor constructing the scalars is found:

The implicitly-defined default constructor performs the set of initializations of the class that would be performed by a user-written default constructor for that class with no ctor-initializer (15.6.2) and an empty compound-statement.

This actually means that everything with a default constructor must have be initialized as-if all the values were listed out as:

int val{};

So the question is, what is the default initialization of an integer initialized with an empty compound-statement? I'm assuming all other compilers determine that to be 0.

[LADSoft]

in that case val should be initialized to zero because it is a form of explicit initialization. but the definition of __a_value does not have any kind of initializer whatsoever????? So that isn't the current issue...

[chuggafan]

It doesn't have an initializer but is treated as if it does for an implicitly-defined default constructor.

[LADSoft]

the problem I'm having is that this sounds like such a fundamental rule that would have breaking implications if anyone actually relied on it happening, and yet none of the compilers I have access to implements this for scalars. Not even clang. Structured members, yes. But not scalars...

[chuggafan]

I remember you said something about making a custom testcase? Do you have it on hand so I can analyze the binary output and see what the compilers output is? All I can tell is clang seems to be initializing the value inside of the default constructor to 0 and that the default defaulted constructor is what's being called in that case.

I can only think that the two statements are supposed to behave exactly the same way and that clang does in fact do this.

[chuggafan]

Rereading it AGAIN, the part I was quoting you're right about at least, it's meant that the default constructor is implicitly defined to look like:

constructor() {}

Sigh, I now need to search for the paragraph where initialization of subobjects is considered....

[chuggafan]

Found something on aggregates:

8 If there are fewer initializer-clauses in the list than there are elements in a non-union aggregate, then each
element not explicitly initialized is initialized as follows:
(8.1) — If the element has a default member initializer (12.2), the element is initialized from that initializer.
(8.2) — Otherwise, if the element is not a reference, the element is copy-initialized from an empty initializer
list (11.6.4).
(8.3) — Otherwise, the program is ill-formed

So for non-union aggregates being initialized in an initializer list these rules are followed...

I do see references when defining a defaulted default constructor as deleted when non-union subobjects with no trivial default constructor and no variants have a "default member initializer", which is a "brace-or-equal-initializer" expression.

I also see triviality being determined by presence of the default member initializer...

The likely paragraph I've found so far that contains the "best" evidence as to behavior is:

9 In a non-delegating constructor, if a given potentially constructed subobject is not designated by a meminitializer-id (including the case where there is no mem-initializer-list because the constructor has no
ctor-initializer), then
(9.1) — if the entity is a non-static data member that has a default member initializer (12.2) and either
(9.1.1) — the constructor’s class is a union (12.3), and no other variant member of that union is designated
by a mem-initializer-id or
(9.1.2) — the constructor’s class is not a union, and, if the entity is a member of an anonymous union, no
other member of that union is designated by a mem-initializer-id,
the entity is initialized from its default member initializer as specified in 11.6;
(9.2) — otherwise, if the entity is an anonymous union or a variant member (12.3.1), no initialization is
performed;
(9.3) — otherwise, the entity is default-initialized (11.6).

The default-initialization is:

7 To default-initialize an object of type T means:
(7.1) — If T is a (possibly cv-qualified) class type (Clause 12), constructors are considered. The applicable
constructors are enumerated (16.3.1.3), and the best one for the initializer () is chosen through overload
resolution (16.3). The constructor thus selected is called, with an empty argument list, to initialize the
object.
(7.2) — If T is an array type, each element is default-initialized.
(7.3) — Otherwise, no initialization is performed.

Which then brings the question back into sight: Where the hell does it say that the scalar should be zero-initialize'd instead of default-initialize'd?

I'm assuming this is done for the sake of simplicity but clang having used it for their library might yet still give me doubts. Either way: Trying to read this standard and find something seemingly simple is a goddamn hot mess.

[chuggafan]

I do believe that once this is done being worked on, there should be an effort into making the libcxx tests part of our regular appveyor tests, or at least run somewhat regularly so that we have a more comprehensive testing regime.

[LADSoft]

yeah i was thinking about adding a new build to the appveyor, that runs the libcxx tests. We wouldn't have to recompile the compiler three times, just run the VS version against the libcxx tests...

[LADSoft]

i started working on the algorithms tests. At first only 4 passed. Now 111 are passing, 35 failing.

[LADSoft]

im looking at solely the 'algorithm' tests right now. 117 passes 29 failures at this point...

[LADSoft]

the numbers went way down on the algorithm tests while i was working on constexpr. But they came back up with some effort and now we are at 125 passes, 21 failures. But there are still two files failing related to constexpr issues at this point, some problems with minmax().

[chuggafan]

I asked a question over on r/cpp re: what's going on with the atomic constructor, it turns out that it is standard behavior and I was being extremely straight-tracked in what I was looking for, so I never checked the initialization type.

The problem was that it's actually value-initialization, http://eel.is/c++draft/dcl.init.general#15.4.sentence-1 and value initialization can zero-initialize if there's a non-trivial default constructor. http://eel.is/c++draft/dcl.init.general#8.1.2

So it is in fact behavior we need to emulate here....

[LADSoft]

ok that is the trick I wanted to see lol! As this is somewhat involved (it also involves having potentially two default constructors) I've made issue #650 specifically for it.

[chuggafan]

Sounds good, Yhea, I should of thought of working backwards: Why zero initialization -> What type of initializations do zero-init -> Is this one of those cases instead of just thinking this was a specific init type and moving forwards.

[LADSoft]

4 of the atomic tests are failing 22 of the algorithm tests are failing (mostly in the sorting support)

I'll start back in on working on these now...

[LADSoft]

now it compiles the atomic and algorithm tests properly. I'm going to have a release and then get back to working on tests...

[GitMensch]

Sounds good - and this is the last open issue for the 3.5 milestone (so far), too.

[LADSoft]

over the last couple of days i fixed all the crashes in the 'container' tests. Now we are down to 812 failures (out of 1177 tests). All but two are parser failures... good news is I scanned through some of the test results and it looks like there may be relatively few common themes for things that are broken.

[LADSoft]

making progress, the container tests are down to 640 failures now. It is worth noting there is something wrong with code generation because I get different results depending on whether the compiler is compiled with itself... but at least it compiles itself and the normal test suite as well as ever.

[chuggafan]

After the container tests are done I think a good thing to work on would be the threading tests, once that's down I can rewrite OMake to use the std:: threading system internally so that I don't have to deal with the CRITICAL_SECTION stuff... (That, and it seems that there's an issue re: testing where even enabling debug mode will prevent compiles for some reason... and crashes are happening despite the debug option being ticked off where I can't see what's causing the crash or where)

[LADSoft]

yeah, there may still be issues with the constexpr handling... which is mostly (but not entirely) turned of when you select 'debug' mode. I fixed a crash related to that last night. A 'class' argument to the function which had been allocated as an argument on the stack was used even though the function call had been elided by the constexpr handling.

I'm hoping things will be a little better after I get through all the bugs with the container stuff. Which is coming along ok, there are a lot of repeat problems so fixing one thing has a moderate impact...

A lot of times I use a combination of MAP files and assembly language output to debug the executables. And sometimes I sprinkle the source code with 'int3' so I can stop in a specific place and I can get a general idea of whether the code is doing what I think it should at that point. I haven't been shy about putting int3 in library headers, or going for finer detail and putting it in an assembly language output file then assembling the file lol... Not as good as a debugger maybe but effective enough. I use an old borland debugger from back in the 90's still (it is barely functional on windows 10 lol) because it is easy to use... But sometimes when its age showed I've loaded up executables in visual studio. That works reasonably well too just harder to get set up.

I figure if we ever get name mangling/pdb handling in there visual studio will become a preferred way for me to debug runtime issues... because I won't have to rely so much on assembly language output files.

I'm trying to figure out what is wrong when the tests use custom allocators with and right now. I got to the point of deciding the pointer to the allocated memory is getting lost last night but had to call it quits...

[LADSoft]

575 failures left on the container tests. About halfway there. Again there are duplicates in some of the upcoming tests... some of it will be easy enough to deal with.

Have to figure out what to do with typeinfo though... probably will just rename our typeinfo and include it in the main typeinfo file when ORANGEC is defined so that I don't have to rewrite the RTTI stuff... there is a question of where to locate it though as it needs to be accessible to the tests.

[LADSoft]

Merry Christmas all!

I've recently fixed some newly introduced problems with compiling the algorithms tests, and after days of fighting to get the 'noexcept' processing to work better and some other stuff the container tests are down to 439 remaining failures.

[LADSoft]

bug fixes are like pulling teeth right now.... I worked over twenty hours on a single bug sigh. But we are down to 371 remaining failures in the containers tests at this point. Out of 1177 tests. Only 18 of those are compiler crashes...

[chuggafan]

WOOOOOT! Question: do you have any way that you specifically generate the map files while the compiles are happening or are you pulling out each failing test case and running the compiler separately to generate the map files for debugging?

I'm planning on starting a bit on some of the multithreading stuff.

[LADSoft]

I haven't actually started debugging broken tests yet... i'm concentrating primarily on getting all the compile errors cleaned up right now. I do know that some tests throw an assert and others just crash and there is one that sometimes hangs, but haven't been too concerned beyond cleaning up compile errors that just make it so tests aren't even generated. Right now cleaning up one or two errors results in a handful of new passing tests...

But as far as generating map files automagically, one could modify the first line of lit.site.cfg like so:

config.cxx_under_test           = "c:/orangec/bin/occ.exe -pl-mx"

it would probably put them in whatever temp directory it puts the executables...

tbh when I'm debugging tests I usually don't debug the tests directly but instead try to make a much smaller example that exhibits the same problem. Makes it much easier to debug especially if you can figure out a way to get rid of all the standard template headers :smile:

[GitMensch]

I'm a bit confused - shouldn't we define _LIBCPP_HAS_NO_INT128 - or has OrangeC support for 128 bit integers?

[chuggafan]

You would be correct that we do not support 128-bit integers, I haven't seen that flag yet though

[GitMensch]

https://github.com/LADSoft/OrangeC/search?q=int128

[chuggafan]

https://github.com/LADSoft/OrangeC/blob/ec236405d20c3f7afe0eb611603a8825c1f51158/src/clibs/cpp/libcxx/include/__config#L891-L893 It auto-defines itself apparently

[LADSoft]

im stopping on the container tests briefly to work on the thread tests: of 261 tests 96 currently fail...

[LADSoft]

making progress on the thread tests. Down to 24 failures...

[LADSoft]

down to one on the threads tests. I'm opting not to fix it because after evaluation I think it couldn't possibly work the way libcxx is written. Problem being it bails in a custom-written operator new() at a time when operator new() has already been used once without it being cached in an object, and then expects the results first call to new() to be cleaned up by the exception handling mechanism. Then asserts when the cleanup doesn't occur.

[LADSoft]

the containers tests got seriously broken while I fixed the thread tests. They only got about a fifth of the way through and then the test manager hung. Tonite I got them to complete again, we've gained a few test failures: up from 371 to 388. Not too bad I guess given that in exchange all but one of the thread tests passes... I'll be back to this this weekend...

[chuggafan]

🤔 I don't think you meant to close this

[LADSoft]

oops

[LADSoft]

making progress: I cleaned a bunch of stuff up and we are down to 313 failures on the container tests.

[LADSoft]

down to 164 failures on the containers tests tonite. That is 855 passing, 164 failing, 158 not applicable.

[LADSoft]

down to 114 failures as of yesterday...