GCC14 overly sensitive to exact SDK version used to build it

[cjones051073]

See discussion at

https://trac.macports.org/ticket/70331

MacPorts GCC14 is built using your tag gcc-14.1-darwin-r1

Our build bots provide binary tarball installations of GCC, which where built using

macOS Sonoma v14.2.1 (23C71) Xcode v15.2 (15C500b) Apple clang version 15.0.0 (clang-1500.1.0.2.5)

It appears that users that have slightly newer Xcode/CLT installed than this (I personally have the latest, so Xcode 15.4 and CLT 15.3) find that GCC installed using this binary install does not work properly. e.g.

> cat test.cc        
#include <iostream>
int
main(int argc, char * argv[]){
}

 > g++-mp-14 ./test.cc 
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/wchar.h:90,
                 from /opt/local/include/gcc14/c++/cwchar:44,
                 from /opt/local/include/gcc14/c++/bits/postypes.h:40,
                 from /opt/local/include/gcc14/c++/iosfwd:42,
                 from /opt/local/include/gcc14/c++/ios:40,
                 from /opt/local/include/gcc14/c++/ostream:40,
                 from /opt/local/include/gcc14/c++/iostream:41,
                 from ./test.cc:1:
/opt/local/lib/gcc14/gcc/arm64-apple-darwin23/14.1.0/include-fixed/stdio.h:383:153: error: macro "__API_AVAILABLE4" requires 5 arguments, but only 4 given
  383 | FILE *fmemopen(void * __restrict __buf, size_t __size, const char * __restrict __mode) __API_AVAILABLE(macos(10.13), ios(11.0), tvos(11.0), watchos(4.0));
      |                                                                                                                                                         ^
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/Availability.h:177,
                 from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/wchar.h:72:
/opt/local/lib/gcc14/gcc/arm64-apple-darwin23/14.1.0/include-fixed/AvailabilityInternal.h:257:13: note: macro "__API_AVAILABLE4" defined here
  257 |     #define __API_AVAILABLE4(arg0,arg1,arg2,arg3,arg4) __API_A(arg0) __API_A(arg1) __API_A(arg2) __API_A(arg3) __API_A(arg4)
      |             ^~~~~~~~~~~~~~~~

Full error here gcc.error.txt

Full log for the GCC build can be found here

https://build.macports.org/builders/ports-14_arm64-builder/builds/35885/steps/install-port/logs/stdio

Other GCC versions we provide in MacPorts, for instance gcc13 (based on your gcc-13.3-darwin-r0 tag) does not appear to have this problem.

[cjones051073]

b.t.w. It would appear this is not unique to MacPorts

https://stackoverflow.com/questions/78452523/g-works-but-g-14-produces-a-lot-of-errors-on-macos

[iains]

please can you show the directory listing of: /path/to/install/lib/gcc/arm64-apple-darwin23/14.1.0/include-fixed/

It appears that (somehow, I do not know how yet) you have broken the configure of the target triple.

The target triples for macOS Arm64 are aarch64-apple-darwinNN not arm64-apple-darwinNN .. I did challenge this when I started the port - but it had already been agreed with the configure folks and Jeremy (AFAIR).

[iains]

that is ( to be clear ) -arch arm64 is correct (and used by the toolchain internally) but the triples are aarch64-apple-darwinNN

[iains]

That might be the problem (defeating the fixincludes detection of the platform) - OTOH it might be the general issue that we do bake some SDK information into a binary distribution - a problem that I'm trying hard to get around; but one that is very difficult to solve in some case, since the SDKs contain non-standard-conforming code in places.

[cjones051073]

 > ls -lth /opt/local/lib/gcc14/gcc/arm64-apple-darwin23/14.1.0/include-fixed
total 64
drwxr-xr-x  4 macports  wheel   128B Jul  8 09:24 os
drwxr-xr-x  3 macports  wheel    96B Jul  8 09:24 objc
drwxr-xr-x  3 macports  wheel    96B Jul  8 09:24 dispatch
-rw-r--r--  1 macports  wheel   750B Jul  3 21:29 README
-rw-r--r--  1 macports  wheel    25K Jul  3 20:59 AvailabilityInternal.h
-rw-r--r--  1 macports  wheel    32K Jul  3 20:59 math.h
-rw-r--r--  1 macports  wheel   5.8K Jul  3 20:59 stdint.h
-rw-r--r--  1 macports  wheel    17K Jul  3 20:59 stdio.h

[cjones051073]

Note https://stackoverflow.com/questions/78452523/g-works-but-g-14-produces-a-lot-of-errors-on-macos is using the Homebrew installation, so whatever is broken it appears broken for them as well.

[iains]

thanks; could you now attached the include-fixed/AvailabilityInternal.h?

Note https://stackoverflow.com/questions/78452523/g-works-but-g-14-produces-a-lot-of-errors-on-macos is using the Homebrew installation, so whatever is broken it appears broken for them as well.

Yeah, it could be the second issue - that SDK information does end up (in the fixed includes) in the binary distribution. I will try to take a look this evening .. just speculating from the office at the moment :)

[cjones051073]

Also note, if I build locally myself, against XCode/CLT 15.4/15.3 all is fine. The issue is specifically when a binary build against Xcode 15.2 is used with the newer SDK.

[cjones051073]

Thanks. My suspicion is indeed that second possibility, that too much SDK specific info is getting baked into GCC.

[iains]

@fxcoudert -- do you have any analysis yet?

[iains]

thanks; could you now attached the include-fixed/AvailabilityInternal.h?

Note https://stackoverflow.com/questions/78452523/g-works-but-g-14-produces-a-lot-of-errors-on-macos is using the Homebrew installation, so whatever is broken it appears broken for them as well.

Yeah, it could be the second issue - that SDK information does end up (in the fixed includes) in the binary distribution. I will try to take a look this evening .. just speculating from the office at the moment :)

but you still need to fix the first issue - the toolchain is not set up to expect "arm64-apple-darwin" triples (I don't know if it even works reliably with clang - at one stage I went through an exhaustive check of what the -cc1 lines resulted from each triple passed to the clang driver).

[cjones051073]

Thanks, I'll change this the next time I push an update to the ports in MacPorts (I don't really want to force a full rebuild on our buildbots for this on its own).

[iains]

Also note, if I build locally myself, against XCode/CLT 15.4/15.3 all is fine. The issue is specifically when a binary build against Xcode 15.2 is used with the newer SDK.

Do you happen to know what the CLT version is there? (I have a setup that allows me to rewind/use earlier versions - so that I could do a build with an earlier one).

NOTE: later this week we will have an RC1 for gcc-11.5 and once that happens my testing resources will be tied up until the release is out.

[cjones051073]

Also note, if I build locally myself, against XCode/CLT 15.4/15.3 all is fine. The issue is specifically when a binary build against Xcode 15.2 is used with the newer SDK.

Do you happen to know what the CLT version is there? (I have a setup that allows me to rewind/use earlier versions - so that I could do a build with an earlier one).

NOTE: later this week we will have an RC1 for gcc-11.5 and once that happens my testing resources will be tied up until the release is out.

from the build log https://build.macports.org/builders/ports-14_arm64-builder/builds/35885/steps/install-port/logs/stdio

DEBUG: macOS 14.2.1 (darwin/23.2.0) arch arm
DEBUG: MacPorts 2.9.3
DEBUG: Xcode 15.2, CLT 15.1.0.0.1.1700200546
DEBUG: SDK 14

[cjones051073]

b.t.w. from the build configure it is true we pass --build=arm64-apple-darwin23 and I will change this, but the configuration appears to translate this to

checking build system type... aarch64-apple-darwin23
checking host system type... aarch64-apple-darwin23
checking target system type... aarch64-apple-darwin23

so it would appear to handle this OK.

[fxcoudert]

Yes, the fixincluded headers leak some SDK info into the installed GCC, and when Apple makes incompatible changes this means the GCC builds become non-portable across SDK versions.

There is no simple solution. If the macOS SDK headers were standard-conforming, the issue wouldn't arise.

[cjones051073]

This the first time we have run into this though, so the current issue is particularly bad with GCC 14.1

[fxcoudert]

This the first time we have run into this though, so the current issue is particularly bad with GCC 14.1

One thing I don't understand, though, is why AvailabilityInternal.h get fixed in your build. In Iains' gcc-14.1-darwin-r1 tag, this commit should prevent the issue: https://github.com/iains/gcc-14-branch/commit/75ff8c390327ac693f6a1c40510bc0d35d7a1e22

[iains]

This the first time we have run into this though, so the current issue is particularly bad with GCC 14.1

One thing I don't understand, though, is why AvailabilityInternal.h get fixed in your build. In Iains' gcc-14.1-darwin-r1 tag, this commit should prevent the issue: 75ff8c3

I'm wondering if this is related to the wrong triple - but I guess maybe not if the HB one is failing the same way.

This the first time we have run into this though, so the current issue is particularly bad with GCC 14.1

One thing I don't understand, though, is why AvailabilityInternal.h get fixed in your build. In Iains' gcc-14.1-darwin-r1 tag, this commit should prevent the issue: 75ff8c3

In fact, it should already have been fixed in r0 .. and the r1 is actually a NOP.

(later) I will try a build using the MacOSX14.2.sdk and then see if that causes issues when used with the MacOSX14.4 one

[cjones051073]

This the first time we have run into this though, so the current issue is particularly bad with GCC 14.1

One thing I don't understand, though, is why AvailabilityInternal.h get fixed in your build. In Iains' gcc-14.1-darwin-r1 tag, this commit should prevent the issue: 75ff8c3

I have no idea what these fixes are doing, but you can see it happening in

https://build.macports.org/builders/ports-14_arm64-builder/builds/35885/steps/install-port/logs/stdio

Applying darwin_availabilityinternal to AvailabilityInternal.h
Fixed:  AvailabilityInternal.h

[cjones051073]

This the first time we have run into this though, so the current issue is particularly bad with GCC 14.1

One thing I don't understand, though, is why AvailabilityInternal.h get fixed in your build. In Iains' gcc-14.1-darwin-r1 tag, this commit should prevent the issue: 75ff8c3

I'm wondering if this is related to the wrong triple - but I guess maybe not if the HB one is failing the same way.

I have pushed an update to our gcc12 and gcc13 builds changing the triple, but have held off doing if for gcc14 until more was known about the issue here. If you think its worth a try though I can push this. It will be short while before the buildbots get through all these builds though.

[iains]

This the first time we have run into this though, so the current issue is particularly bad with GCC 14.1

One thing I don't understand, though, is why AvailabilityInternal.h get fixed in your build. In Iains' gcc-14.1-darwin-r1 tag, this commit should prevent the issue: 75ff8c3

I'm wondering if this is related to the wrong triple - but I guess maybe not if the HB one is failing the same way.

I have pushed an update to our gcc12 and gcc13 builds changing the triple, but have held off doing if for gcc14 until more was known about the issue here. If you think its worth a try though I can push this. It will be short while before the buildbots get through all these builds though.

No need from my perspective - I need to figure out what's going wrong - at least, to repeat it outside of your infra.

[iains]

I have now checked gcc-14.1-darwin-r1 on macOS14.5 with CLT 15.3 (MacOSX14.4.sdk) and, indeed it does not generate a fixed include for that file (so working as intended). Later, I will try with the MacOSX14.2.sdk...

[cjones051073]

This the first time we have run into this though, so the current issue is particularly bad with GCC 14.1

One thing I don't understand, though, is why AvailabilityInternal.h get fixed in your build. In Iains' gcc-14.1-darwin-r1 tag, this commit should prevent the issue: 75ff8c3

As far as I can tell, the homebrew builds have the same fixincludes for this file

https://stackoverflow.com/questions/78452523/g-works-but-g-14-produces-a-lot-of-errors-on-macos

/usr/local/Cellar/gcc/14.1.0/lib/gcc/current/gcc/x86_64-apple-darwin23/14/include-fixed/AvailabilityInternal.h:254:13: note: macro "__API_AVAILABLE0" defined here
  254 |     #define __API_AVAILABLE0(arg0) __API_A(arg0)
      |             ^~~~~~~~~~~~~~~~

[iains]

I have now checked gcc-14.1-darwin-r1 on macOS14.5 with CLT 15.3 (MacOSX14.4.sdk) and, indeed it does not generate a fixed include for that file (so working as intended). Later, I will try with the MacOSX14.2.sdk...

.. which produces the same result; that should not be surprising - we removed the code that did the substitution; there is no reference to AvailabilityInternal.h or darwin_availabilityinternal in fixincludes in my gcc-14-1-darwin tree checked out to 0b3ebcb06ad6.

So .. it's quite amazing that your build says :

Applying darwin_availabilityinternal to AvailabilityInternal.h
Fixed:  AvailabilityInternal.h

Sorry, I did not yet have a chance to go through the build log .. it's been a hectic day.

[ryandesign]

there is no reference to AvailabilityInternal.h or darwin_availabilityinternal in fixincludes in my gcc-14-1-darwin tree checked out to 0b3ebcb.

But there is in the gcc-14.1.0.tar.xz tarball from gnu.org that MacPorts is using:

% tar xJf gcc-14.1.0.tar.xz
% cd gcc-14.1.0
% grep -i availabilityinternal -r .
./MD5SUMS:e629d76c4229bd51744b1e2deab205be  fixincludes/tests/base/AvailabilityInternal.h
./fixincludes/ChangeLog:    * inclhack.def (darwin_availabilityinternal, darwin_os_trace_1)
./fixincludes/ChangeLog:    * tests/bases/AvailabilityInternal.h: New file.
./fixincludes/fixincl.x: *  Description of Darwin_Availabilityinternal fix
./fixincludes/fixincl.x:tSCC zDarwin_AvailabilityinternalName[] =
./fixincludes/fixincl.x:     "darwin_availabilityinternal";
./fixincludes/fixincl.x:tSCC zDarwin_AvailabilityinternalList[] =
./fixincludes/fixincl.x:  "AvailabilityInternal.h\0";
./fixincludes/fixincl.x:tSCC* apzDarwin_AvailabilityinternalMachs[] = {
./fixincludes/fixincl.x:tSCC zDarwin_AvailabilityinternalSelect0[] =
./fixincludes/fixincl.x:#define    DARWIN_AVAILABILITYINTERNAL_TEST_CT  1
./fixincludes/fixincl.x:static tTestDesc aDarwin_AvailabilityinternalTests[] = {
./fixincludes/fixincl.x:  { TT_EGREP,    zDarwin_AvailabilityinternalSelect0, (regex_t*)NULL }, };
./fixincludes/fixincl.x: *  Fix Command Arguments for Darwin_Availabilityinternal
./fixincludes/fixincl.x:static const char* apzDarwin_AvailabilityinternalPatch[] = {
./fixincludes/fixincl.x:    DARWIN_AVAILABILITYINTERNAL_FIXIDX,
./fixincludes/fixincl.x:  {  zDarwin_AvailabilityinternalName,    zDarwin_AvailabilityinternalList,
./fixincludes/fixincl.x:     apzDarwin_AvailabilityinternalMachs,
./fixincludes/fixincl.x:     DARWIN_AVAILABILITYINTERNAL_TEST_CT, FD_MACH_ONLY | FD_SUBROUTINE,
./fixincludes/fixincl.x:     aDarwin_AvailabilityinternalTests,   apzDarwin_AvailabilityinternalPatch, 0 },
./fixincludes/tests/base/AvailabilityInternal.h:    "fixinc/tests/inc/AvailabilityInternal.h"
./fixincludes/tests/base/AvailabilityInternal.h:#if defined( DARWIN_AVAILABILITYINTERNAL_CHECK )
./fixincludes/tests/base/AvailabilityInternal.h:#endif  /* DARWIN_AVAILABILITYINTERNAL_CHECK */
./fixincludes/inclhack.def: *  macOS 10.12 <AvailabilityInternal.h> uses __attribute__((availability))
./fixincludes/inclhack.def:    hackname  = darwin_availabilityinternal;
./fixincludes/inclhack.def:    files     = AvailabilityInternal.h;

[ryandesign]

As far as I can tell, the homebrew builds have the same fixincludes for this file

https://stackoverflow.com/questions/78452523/g-works-but-g-14-produces-a-lot-of-errors-on-macos

That question was asked in May. Homebrew applied a patch to fix it in June: https://github.com/Homebrew/homebrew-core/commit/61581e69c86abc0c6b816a06d78e4320f5355335

[iains]

so, somehow, the patch set on the arm64 branch is perhaps mis-applied (my branch is based off cd0059a19763 which is the [upstream] gcc-14.1.0 tag for the release ). Therefore, we should be on the same base - and commit 75ff8c390327 should remove the handling. Or, perhaps, you are applying a patch based on gcc-14.1-darwin-r0 which does not include that fix?

[ryandesign]

The MacPorts gcc14 port includes an unreviewable 22,772-line patchfile which says **_The current release is GCC-14.1-darwin-r0. (May 2024)_**.

[iains]

The MacPorts gcc14 port includes an unreviewable 22,772-line patchfile which says **_The current release is GCC-14.1-darwin-r0. (May 2024)_**.

that would explain it - the r1 was released to fix this exact problem.

[ryandesign]

So @cjones051073 this does not appear to be accurate:

MacPorts GCC14 is built using your tag gcc-14.1-darwin-r1

And @iains this does not appear to be accurate:

In fact, it should already have been fixed in r0 .. and the r1 is actually a NOP.

[iains]

So @cjones051073 this does not appear to be accurate:

MacPorts GCC14 is built using your tag gcc-14.1-darwin-r1

And @iains this does not appear to be accurate:

In fact, it should already have been fixed in r0 .. and the r1 is actually a NOP.

Indeed you are correct - I misremembered - the r1 that is a NOP is on the 13.3 branch which was released only a week or two later (and that I just checked to avoid bad memory striking again).

[cjones051073]

Indeed, it would appear somehow I generated the patchfile we use from the r0 tag, and not the r1 one. I apologies for that, not sure how I missed this. Anyway, am testing an update now locally and will push once it appears OK for me.

you can see the changes to the patch file, going from r0 to r1 here

https://github.com/cjones051073/macports-ports/commit/946f6c9ec4f2fd90fa7e3ec93a05fe6983c1e40f

and indeed it looks like its all relating to this availability stuff.

[iains]

for the record, the patch is present on 13.3-darwin-r0/1, 12.4-darwin-r0 and will be on the soon-to-be-released 11.5-darwin-r0.

[cjones051073]

Thanks for that info.

For the record, I have now tested MacPorts buildbot build binary installation on my macOS14-arm machine here and it is now appears OK.

[mouse07410]

Also, I see problems with the headers gcc14/libc++/include, which in Macports are in /opt/local/libexec/gcc14/libc++/include.

Specifically, file cstdlib lost an important check, and requires the following patch:

diff -uw /opt/local/libexec/gcc14/libc++/include/c++/v1/cstdlib.orig /opt/local/libexec/gcc14/libc++/include/c++/v1/cstdlib
--- /opt/local/libexec/gcc14/libc++/include/c++/v1/cstdlib.orig 2024-07-09 13:43:11
+++ /opt/local/libexec/gcc14/libc++/include/c++/v1/cstdlib  2024-07-10 09:41:58
@@ -142,7 +142,7 @@
 using ::mbstowcs _LIBCPP_USING_IF_EXISTS;
 using ::wcstombs _LIBCPP_USING_IF_EXISTS;
 #endif
-#if !defined(_LIBCPP_CXX03_LANG)
+#if !defined(_LIBCPP_CXX03_LANG) && defined(_LIBCPP_HAS_QUICK_EXIT)
 using ::at_quick_exit _LIBCPP_USING_IF_EXISTS;
 using ::quick_exit _LIBCPP_USING_IF_EXISTS;
 #endif

---

File libc++/include/c++/v1/__filesystem/path.h invokes a GCC14 compiler complaint:

g++ -stdlib=libc++ -pipe -DNDEBUG -g2 -Wa,-q -fPIC -pthread -fno-common -pipe -std=gnu++20 -O3 -march=native -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -c dll.cpp
In file included from /opt/local/libexec/gcc14/libc++/include/c++/v1/__filesystem/filesystem_error.h:15,
                 from /opt/local/libexec/gcc14/libc++/include/c++/v1/__filesystem/directory_entry.h:20,
                 from /opt/local/libexec/gcc14/libc++/include/c++/v1/filesystem:539,
                 from /opt/local/libexec/gcc14/libc++/include/c++/v1/fstream:199,
                 from files.h:16,
                 from dll.h:27,
                 from dll.cpp:6:
/opt/local/libexec/gcc14/libc++/include/c++/v1/__filesystem/path.h: In instantiation of 'std::__1::__fs::filesystem::path::_EnableIfPathable<_Source> std::__1::__fs::filesystem::path::append(const _Source&) [with _Source = std::__1::basic_string<char>]':
/opt/local/libexec/gcc14/libc++/include/c++/v1/__filesystem/path.h:534:52: error: use of built-in trait '__remove_pointer(typename std::__1::decay<_Tp>::type)' in function signature; use library traits instead
  534 |   _LIBCPP_HIDE_FROM_ABI _EnableIfPathable<_Source> append(const _Source& __src) {
      |                                                    ^~~~~~

Alas, I don't know what's the right way to address it - help is welcome.

[iains]

1/ please start new issues for things that are unrelated.

2/ I don't think this is arm64-related.

thanks to @jwakely for pointing to the following upstream LLVM commit which most likely fixes it - I guess you need to back port that along with the quick_exit fix [that is unfortunate, that upstream has assumed that either one has the function or the support for attribute using].

commit cb7a03b41fff563c0cbb5145eed09f9b17edf9e2
Author: Nikolas Klauser <nikolasklauser@berlin.de>
Date:   2024-06-01 11:20:41 +0100

     [libc++] Fix failures with GCC 14 (#92663)

    Fixes #91831

[jwakely]

https://github.com/llvm/llvm-project/commit/cb7a03b41fff563c0cbb5145eed09f9b17edf9e2#diff-85e955a4afe2d8205f7527c3374fbf66a958bccfe28608a15e10f109f5ca56f1

This part, specifically

[iains]

where are we with this issue?

(AFAIK we understand that the macOS SDK has changed between macOS14 and macOS15 in incompatible ways, when using fixed includes - which we still need to do).

The solution is to use a compiler with the SDK it was built with; this is relatively easy with the CLT installed since both macOS14 and 15 SDKs are available. It might be less easy for folks using the SDKs bundled with Xcode.app.

An alternative, of course, would be to hire someone to modify GCC to accept all of the macOS SDK clang extensions.