Problems with C include path: need --with-build-root

[simonjwright]

(this arises from changes I made in response to this note on issue #3, but I think it’s worth its own topic).

The problem is that, unless I configure with --with-build-root=$SDKROOT, the built compiler doesn’t include $prefix/include in the include search path. (I should add that I and my colleagues are still using the run-time determination of XC/CLT via --with-specs= -- BZ outstanding)

The source is gcc-14-branch, tag gcc-14.1-darwin-r1.

$prefix is /Volumes/Miscellaneous3/aarch64/gcc-14.1.0-1-aarch64.

I'm compiling gcc -v hello.c.

If I configured gcc with just

--with-sysroot=$SDKROOT

then I see

...
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/local/include"
ignoring duplicate directory "/Volumes/Miscellaneous3/aarch64/gcc-14.1.0-1-aarch64/bin/../lib/gcc/../../lib/gcc/aarch64-apple-darwin21/14.1.0/include-fixed"
ignoring nonexistent directory "/Volumes/Miscellaneous3/aarch64/gcc-14.1.0-1-aarch64/bin/../lib/gcc/../../lib/gcc/aarch64-apple-darwin21/14.1.0/../../../../aarch64-apple-darwin21/include"
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:
 /Volumes/Miscellaneous3/aarch64/gcc-14.1.0-1-aarch64/bin/../lib/gcc/aarch64-apple-darwin21/14.1.0/include
 /Volumes/Miscellaneous3/aarch64/gcc-14.1.0-1-aarch64/bin/../lib/gcc/aarch64-apple-darwin21/14.1.0/include-fixed
 /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include
 /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks
End of search list.

and there is no $prefix/include. The third 'nonexistent directory' is $prefix/aarch64-apple-darwin21/include, which would be appropriate for a cross-compiler. I only specified --build, not --host or --target.

If I configure with

--with-build-sysroot=$SDKROOT
--with-sysroot=

then I get

...
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/local/include"
ignoring nonexistent directory "/Volumes/Miscellaneous3/aarch64/gcc-14.1.0-1-aarch64/lib/gcc/aarch64-apple-darwin21/14.1.0/../../../../aarch64-apple-darwin21/include"
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:
 /Volumes/Miscellaneous3/aarch64/gcc-14.1.0-1-aarch64/lib/gcc/aarch64-apple-darwin21/14.1.0/include
 /Volumes/Miscellaneous3/aarch64/gcc-14.1.0-1-aarch64/include
 /Volumes/Miscellaneous3/aarch64/gcc-14.1.0-1-aarch64/lib/gcc/aarch64-apple-darwin21/14.1.0/include-fixed
 /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include
 /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks
End of search list.

then there is $prefix/include.

My colleagues are configuring with just --with-build-sysroot=$SDKROOT (i.e. without --with-sysroot=), and also have no problems.

[iains]

this is https://gcc.gnu.org/bugzilla/show_bug.cgi?id=79885

not specific to any of the branches here.

You are breaking what the configure options do by specifying --with-sysroot= (I think) see https://gcc.gnu.org/install/configure.html

my recommendation is "don't do that" .. actually it does not achieve anything different from "--with-sysroot=" on Darwin platforms .. the compiler will still honour --sysroot= and SDKROOT at use-time;

[iains]

My colleagues are configuring with just --with-build-sysroot=$SDKROOT (i.e. without --with-sysroot=), and also have no problems.

I wonder if they are just not hitting anything that needs fixed includes .. that is you should check their installations to see if there are any fixed includes (I'd suspect not)

[simonjwright]

My fixed includes are

$ ls -RF include-fixed
README      math.h      os/     stdio.h
dispatch/   objc/       stdint.h

include-fixed/dispatch:
object.h

include-fixed/objc:
runtime.h

include-fixed/os:
base.h      trace.h

Colleagues, the same.

I've looked at https://gcc.gnu.org/bugzilla/show_bug.cgi?id=79885, I wonder whether the reason we aren't having problems is that --with-specs line? though, on checking all the config.logs, each of them contains --with-build-sysroot=/Library/Developer.

[iains]

I would expect that this behaviour is unchanged by the branches here - we are about to start the release process for gcc-14.2 - would you have time to check your build recipes on the upstream "releases/gcc-14" (of course, that is only applicable to x86_64 at present) ..

[simonjwright]

With releases/gcc-14, results from make check.

This is just acats, gcc, g++, gnat; I've seen other tests reported, but I don't know how to run them.

Configured with:

/Volumes/Miscellaneous3/src/gcc/configure
--prefix=/Volumes/Miscellaneous3/x86_64/gcc-14.1.1-1-x86_64
--enable-languages=c,c++,ada
--build=x86_64-apple-darwin21
--with-build-sysroot=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
--with-specs='%{!sysroot=*:--sysroot=%:if-exists-else(/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk)}'
--with-bugurl=https://github.com/simonjwright/building-gcc-macos-native
--enable-bootstrap

acats

        === acats configuration ===
target gcc is /Volumes/Miscellaneous3/x86_64/14.1.1/gcc/gcc/xgcc -B/Volumes/Miscellaneous3/x86_64/14.1.1/gcc/gcc/
Reading specs from /Volumes/Miscellaneous3/x86_64/14.1.1/gcc/gcc/specs COLLECT_GCC=/Volumes/Miscellaneous3/x86_64/14.1.1/gcc/gcc/xgcc COLLECT_LTO_WRAPPER=/Volumes/Miscellaneous3/x86_64/14.1.1/gcc/gcc/lto-wrapper Target: x86_64-apple-darwin21 Configured with: /Volumes/Miscellaneous3/src/gcc/configure --prefix=/Volumes/Miscellaneous3/x86_64/gcc-14.1.1-1-x86_64 --enable-languages=c,c++,ada --build=x86_64-apple-darwin21 --with-build-sysroot=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk --with-specs='%{!sysroot=*:--sysroot=%:if-exists-else(/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk)}' --with-bugurl=https://github.com/simonjwright/building-gcc-macos-native --enable-bootstrap Thread model: posix Supported LTO compression algorithms: zlib gcc version 14.1.1 20240719 (GCC)
host=aarch64-apple-darwin21
target=x86_64-apple-darwin21
gnatmake is /Volumes/Miscellaneous3/x86_64/14.1.1/gcc/gcc/gnatmake

        === acats Summary ===
# of expected passes        2327
# of unexpected failures    1
*** FAILURES: c250002 
/Volumes/Miscellaneous3/src/gcc/gcc/testsuite/ada/acats/run_all.sh completed at Fri 19 Jul 2024 19:36:42 BST

gcc

Test Run By simon on Fri Jul 19 18:12:12 2024
Native configuration is x86_64-apple-darwin21

        === gcc tests ===

Schedule of variations:
    unix

Running target unix

        === gcc Summary ===

# of expected passes        192406
# of unexpected failures    58
# of unexpected successes   6
# of expected failures      1467
# of unsupported tests      3963
/Volumes/Miscellaneous3/x86_64/14.1.1/gcc/gcc/xgcc  version 14.1.1 20240719 (GCC) 

g++

Test Run By simon on Fri Jul 19 18:12:12 2024
Native configuration is x86_64-apple-darwin21

        === g++ tests ===

Schedule of variations:
    unix

Running target unix

        === g++ Summary ===

# of expected passes        259059
# of unexpected failures    70
# of expected failures      2780
# of unsupported tests      11897
/Volumes/Miscellaneous3/x86_64/14.1.1/gcc/gcc/testsuite/g++/../../xg++  version 14.1.1 20240719 (GCC) 

gnat

Test Run By simon on Fri Jul 19 18:12:12 2024
Native configuration is x86_64-apple-darwin21

        === gnat tests ===

Schedule of variations:
    unix

Running target unix

        === gnat Summary ===

# of expected passes        3483
# of unexpected failures    1
# of expected failures      24
# of unsupported tests      14
/Volumes/Miscellaneous3/x86_64/14.1.1/gcc/gcc/gnatmake version 14.1.1 20240719

[iains]

Those results look reasonable to me - perhaps I should push the latest test branch...

actually, this is a bit suspicious:

host=aarch64-apple-darwin21
target=x86_64-apple-darwin21

Are you intending to build a cross-compiler, or are you intending to bootstrap under rosetta2?

[iains]

perhaps you forgot to arch -x86_64 bash so that you have an x86_64 rosetta2 environment to build in? ( presumably you can substitute your usual shell for bash in the above )

[iains]

from the installation page:

--with-sysroot
--with-sysroot=dir
Tells GCC to consider dir as the root of a tree that contains (a subset of) the root filesystem of the target operating system. Target system headers, libraries and run-time object files will be searched for in there. More specifically, this acts as if --sysroot=dir was added to the default options of the built compiler. The specified directory is not copied into the install tree, unlike the options --with-headers and --with-libs that this option obsoletes. The default value, in case --with-sysroot is not given an argument, is ${gcc_tooldir}/sys-root. If the specified directory is a subdirectory of ${exec_prefix}, then it will be found relative to the GCC binaries if the installation tree is moved.

This option affects the system root for the compiler used to build target libraries (which runs on the build system) and the compiler newly installed with make install; it does not affect the compiler which is used to build GCC itself.

If you specify the --with-native-system-header-dir=dirname option then the compiler will search that directory within dirname for native system headers rather than the default /usr/include.

--with-build-sysroot
--with-build-sysroot=dir
Tells GCC to consider dir as the system root (see --with-sysroot) while building target libraries, instead of the directory specified with --with-sysroot. This option is only useful when you are already using --with-sysroot. You can use --with-build-sysroot when you are configuring with --prefix set to a directory that is different from the one in which you are installing GCC and your target libraries.

This option affects the system root for the compiler used to build target libraries (which runs on the build system); it does not affect the compiler which is used to build GCC itself.

If you specify the --with-native-system-header-dir=dirname option then the compiler will search that directory within dirname for native system headers rather than the default /usr/include.

I would still expect your spec (using --sysroot= ) to override when building target libraries + when installed; Sorry, I'm having a tough time figuring out where the problem is since I have not yet been able to reproducee it ...

(NOTING that we know --with-build-sysroot= does not work properly for generating fixincludes as per the referenced bug)

[iains]

.. needless to say, it would be great to fix this before 14.2 is released - but first we have to figure out where the problem comes from.

[iains]

please could you try https://github.com/iains/gcc-14-branch/tree/gcc-14-2-darwin-pre-0

(I have built this on both x86_64 and aarch64 darwin23 and 21 with all languages).

[iains]

A note in passing:

--with-specs='%{!sysroot=*:--sysroot=%:if-exists-else(/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk)}

Is doing something unsupported - there is no way to tell if it will work - here's why:

Apple's SDKs:

• make use of extensions currently only available in clang
• in some cases choose to include behaviour which is outside of standard(s) that apply

In order to use them with GCC, we apply some pre-headers (the 'fixed includes') that interpose modified headers to account for the extensions or other behaviour that is currently unsupported in GCC.

As time goes on, we have (and continue to) try to support more of the clang extensions - however even if we manage to do all of those, supporting non-standard behaviour is more contentious.

So.. the fixed includes are built on the basis of the SDK that is supplied at the time the compiler is built - by scanning that SDK for known issues and providing the interposed headers.

Those interposed headers quite possibly will not work with a different SDK because:

• in some cases the fixes are no longer required because a newer SDK is already fixed
• in some cases a newer SDK needs other fixes that were not needed in the build-time one so that the new one fails
• in some cases the text of headers has changed and the interposed one is no longer complete

So choosing an arbitrary SDK at compiler run-time with this spec might (or might not) work (the same actually applies to a the one installed in /Library/Developer/CommandLineTools/SDKs/) ..

We are trying hard to minimise the number of fixed includes to make this more likely to succeed .. but it's a slow job against a moving target - and SDKs for older OS versions are not re-released - so we are stuck with fixes there.

[simonjwright]

actually, this is a bit suspicious:

host=aarch64-apple-darwin21
target=x86_64-apple-darwin21

Are you intending to build a cross-compiler, or are you intending to bootstrap under rosetta2?

I just looked at the script (gcc/gcc/testsuite/ada/acats/run_all.sh), and the host= line is generated by gcc -dumpmachine, while the target= line is generated by $GCC -dumpmachine, where GCC="$BASE/xgcc -B$BASE/" (the newly-built compiler).

This is a bootstrap under Rosetta2 - the scripts check the base compiler's architecture is the same as the current shell's.

The base compiler was gcc-13.3.0 built for x86_64.

[iains]

actually, this is a bit suspicious:

host=aarch64-apple-darwin21
target=x86_64-apple-darwin21

Are you intending to build a cross-compiler, or are you intending to bootstrap under rosetta2?

I just looked at the script (gcc/gcc/testsuite/ada/acats/run_all.sh), and the host= line is generated by gcc -dumpmachine, while the target= line is generated by $GCC -dumpmachine, where GCC="$BASE/xgcc -B$BASE/" (the newly-built compiler).

This is a bootstrap under Rosetta2 - the scripts check the base compiler's architecture is the same as the current shell's.

FAOD: if you type arch in the shell you are using for bootstrap - you get i386? I have found that rosetta builds are not reliable otherwise.

(since we have a failure report on a native x86_64 setup, it seems unlikely that this is the actual problem - but it's better to be safe than sorry, right?)

The base compiler was gcc-13.3.0 built for x86_64.

[iains]

when you update your source tree - do you use ./contrib/gcc_update ? if not, do you execute ./contrib/gcc_update --touch to ensure that timestamps on committed dependent build artefacts are OK?

(another possible hypothesis: if the latter is not true, perhaps your build is regenerating some committed artefact that mine is not)

[simonjwright]

This is a bootstrap under Rosetta2 - the scripts check the base compiler's architecture is the same as the current shell's.

FAOD: if you type arch in the shell you are using for bootstrap - you get i386? I have found that rosetta builds are not reliable otherwise.

Yes -- the Makefile fragment is

# what architecture are we running? (will get arm64 or i386)
current_arch = $(shell arch)

# what architecture was the compiler built for?
aarch64 = $(findstring aarch64, $(shell gcc -print-libgcc-file-name))
x86_64  = $(findstring x86_64,  $(shell gcc -print-libgcc-file-name))

# check that the compiler matches the architecture
ifeq ($(current_arch),arm64)
  ifeq ($(aarch64),)
    $(error shell is aarch64 but current gcc is not)
  else
    export ARCH=aarch64
  endif
else
  ifeq ($(current_arch),i386)
    ifeq ($(x86_64),)
      $(error shell is x86_64 but current gcc is not)
    else
      export ARCH=x86_64
    endif
  else
    $(error unexpected architecure $(current_arch), expecting i386)
  endif
endif

when you update your source tree - do you use ./contrib/gcc_update ? if not, do you execute ./contrib/gcc_update --touch to ensure that timestamps on committed dependent build artefacts are OK?

Neither. Though I'm not sure what you mean by 'update your source tree'.

I've taken it that downloading an FSF release will produce a consistent set of files, so I do contrib/download_prerequisites and leave it at that.

With your gcc-14-2-darwin-pre-0, I did git fetch, git switch gcc-14-2-darwin-pre-0, contrib/download_prerequisites. What else should I have done?

[iains]

when you update your source tree - do you use ./contrib/gcc_update ? if not, do you execute ./contrib/gcc_update --touch to ensure that timestamps on committed dependent build artefacts are OK?

Neither. Though I'm not sure what you mean by 'update your source tree'.

I've taken it that downloading an FSF release will produce a consistent set of files, so I do contrib/download_prerequisites and leave it at that.

If it's a tarball - that should be fine, I believe.

With your gcc-14-2-darwin-pre-0, I did git fetch, git switch gcc-14-2-darwin-pre-0, contrib/download_prerequisites. What else should I have done?

./contrib/gcc_update --touch will ensure that the timestamps on built artefacts that are committed to the tree are newer than their dependencies. If that produces no output, then that would indicate this is also not a source of possible differences.

[simonjwright]

please could you try https://github.com/iains/gcc-14-branch/tree/gcc-14-2-darwin-pre-0

(I have built this on both x86_64 and aarch64 darwin23 and 21 with all languages).

(I called it gcc-14.2.0-0, I see it thinks of itself as 14.1.1, sorry for any confusion)

        === acats configuration ===
target gcc is /Volumes/Miscellaneous3/aarch64/14.2.0-0/gcc/gcc/xgcc -B/Volumes/Miscellaneous3/aarch64/14.2.0-0/gcc/gcc/
Reading specs from /Volumes/Miscellaneous3/aarch64/14.2.0-0/gcc/gcc/specs COLLECT_GCC=/Volumes/Miscellaneous3/aarch64/14.2.0-0/gcc/gcc/xgcc COLLECT_LTO_WRAPPER=/Volumes/Miscellaneous3/aarch64/14.2.0-0/gcc/gcc/lto-wrapper Target: aarch64-apple-darwin21 Configured with: /Volumes/Miscellaneous3/src/gcc-14-branch/configure --prefix=/Volumes/Miscellaneous3/aarch64/gcc-14.2.0-0-aarch64 --enable-languages=c,c++,ada --build=aarch64-apple-darwin21 --with-build-sysroot=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk --with-specs='%{!sysroot=*:--sysroot=%:if-exists-else(/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk)}' --with-bugurl=https://github.com/simonjwright/building-gcc-macos-native --enable-bootstrap Thread model: posix Supported LTO compression algorithms: zlib gcc version 14.1.1 20240720 (GCC)
host=aarch64-apple-darwin21
target=aarch64-apple-darwin21
gnatmake is /Volumes/Miscellaneous3/aarch64/14.2.0-0/gcc/gcc/gnatmake

        === acats Summary ===
# of expected passes        2325
# of unexpected failures    3
*** FAILURES: c250002 cb1010a cb1010d 
/Volumes/Miscellaneous3/src/gcc-14-branch/gcc/testsuite/ada/acats/run_all.sh com

        === gcc Summary ===

# of expected passes        213591

# of unexpected failures    66
# of unexpected successes   7
# of expected failures      1744
# of unresolved testcases   6
# of unsupported tests      3108
/Volumes/Miscellaneous3/aarch64/14.2.0-0/gcc/gcc/xgcc  version 14.1.1 20240720 (GCC) 

        === g++ Summary ===

# of expected passes        296153
# of unexpected failures    38
# of unexpected successes   4
# of expected failures      3033
# of unresolved testcases   1
# of unsupported tests      11891
/Volumes/Miscellaneous3/aarch64/14.2.0-0/gcc/gcc/testsuite/g++/../../xg++  version 14.1.1 20240720 (GCC) 

        === gnat Summary ===

# of expected passes        3439
# of unexpected failures    2
# of expected failures      24
# of unsupported tests      35
/Volumes/Miscellaneous3/aarch64/14.2.0-0/gcc/gcc/gnatmake version 14.1.1 20240720

[simonjwright]

./contrib/gcc_update --touch will ensure that the timestamps on built artefacts that are committed to the tree are newer than their dependencies. If that produces no output, then that would indicate this is also not a source of possible differences.

I got

$ contrib/gcc_update --touch
Touching gcc/cstamp-h.in...
Touching gcc/config.in...

Does this mean I need to rebuild?

[iains]

you results seem reasonable to me - it seems to be bootstrapping and working ..

[iains]

the 14.1.1 thing is correct - this will be 14.2 when it's released but is still 14.1.1 now

[iains]

so .. starting from this point - now where do we have a problem?

[simonjwright]

A note in passing:

--with-specs='%{!sysroot=*:--sysroot=%:if-exists-else(/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk)}

Is doing something unsupported - there is no way to tell if it will work - here's why:

Apple's SDKs:

• make use of extensions currently only available in clang
• in some cases choose to include behaviour which is outside of standard(s) that apply

In order to use them with GCC, we apply some pre-headers (the 'fixed includes') that interpose modified headers to account for the extensions or other behaviour that is currently unsupported in GCC.

Anyone who provides a pre-built GCC must face this issue. We could insist that users supply --sysroot=$SDKROOT at every compilation; I don't see what the difference is, they can/will still update XC or CLT, outside our control.

So far the only time this has caused (obvious) issues for us is with the Availability fix (using a compiler built against CLT15 with Availability in the fixed includes when CLT 14 is the one installed).

BTW, exporting SDKROOT doesn't seem to affect the compilers I've built??

[iains]

A note in passing:

--with-specs='%{!sysroot=*:--sysroot=%:if-exists-else(/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk)}

Is doing something unsupported - there is no way to tell if it will work - here's why: Apple's SDKs:

• make use of extensions currently only available in clang
• in some cases choose to include behaviour which is outside of standard(s) that apply

In order to use them with GCC, we apply some pre-headers (the 'fixed includes') that interpose modified headers to account for the extensions or other behaviour that is currently unsupported in GCC.

Anyone who provides a pre-built GCC must face this issue. We could insist that users supply --sysroot=$SDKROOT at every compilation; I don't see what the difference is, they can/will still update XC or CLT, outside our control.

yes, indeed they do, and the users can as you say update ...

.. but asking the user to provide --sysroot= does not solve the problem - the problem is that your SDK might mismatch with theirs; I think the only real workable solution is for you to test which versions of XC your distribution works with and document that as "what's supported" - I believe that distros like Homebrew make similar requirements (@fxcoudert ? any other advice?)

So far the only time this has caused (obvious) issues for us is with the Availability fix (using a compiler built against CLT15 with Availability in the fixed includes when CLT 14 is the one installed).

that should no longer be an issue on any of the release darwin branches, they all have the patches to implement the availability attribute - and the patch that removes the handling from the fixed includes.

BTW, exporting SDKROOT doesn't seem to affect the compilers I've built??

I will take a look at this to double-check but - AFAIR the sysroot priority should be "--sysroot= > SDKROOT > configured value " so if you supply a sysroot= that will override SDKROOT. [we have to choose some priority - and I think we do the same as clang]

[simonjwright]

so .. starting from this point - now where do we have a problem?

My normal build has --with-build-sysroot=$SDKROOT, no --with-sysroot, and my --with-specs line; no problems.

If I build with --with-build-sysroot=$SDKROOT, no --with-sysroot, no --with-specs, I get build errors

• --disable-bootstrap, error while configuring gnattools-native - no library for -liconv.
• --enable-bootstrap - failure, sorry, can't remember where.

If I build with just --with-sysroot=$SDKROOT, the compiler build succeeds but the resulting compiler doesn't include $prefix/include in its search path. This is for both x86_64 on branch releases/gcc-14 and aarch64 on branch gcc-14-2-darwin-pre-0.

So, the builds were configured with

--prefix=...
--enable-languages=c,c++,ada
--build=$ARCH-apple-darwin21
--with-sysroot=$SDKROOT (CLT 15.3)
--with-bugurl=...
--enable-bootstrap

What am I missing?

[simonjwright]

.. but asking the user to provide --sysroot= does not solve the problem

The problem that my --with-specs line addresses is that I don't know whether users will have XC or CLT installed. Asking them to either install the SDK I built against, or set SDKROOT to their SDK/use --sysroot, is going to be off-putting; and I really want to minimise the grief that potential Ada users face. It needs to Just Work.

the problem is that your SDK might mismatch with theirs; I think the only real workable solution is for you to test which versions of XC your distribution works with and document that as "what's supported"

I will action this. Great suggestion.

[iains]

so .. starting from this point - now where do we have a problem?

I'm not sure ...

If I build with just --with-sysroot=$SDKROOT, the compiler build succeeds but the resulting compiler doesn't include $prefix/include in its search path. This is for both x86_64 on branch releases/gcc-14 and aarch64 on branch gcc-14-2-darwin-pre-0.

this is what surprises me - I would not expect the C++ compiler to work without the path to the installed include directory - but clearly it does ...

I'm very tied up with the release and $dayjob .. but i will try to configure as you have done and see what's different.

[iains]

so .. starting from this point - now where do we have a problem?

If I build with just --with-sysroot=$SDKROOT, the compiler build succeeds but the resulting compiler doesn't include $prefix/include in its search path. This is for both x86_64 on branch releases/gcc-14 and aarch64 on branch gcc-14-2-darwin-pre-0.

this is what surprises me - I would not expect the C++ compiler to work without the path to the installed include directory - but clearly it does ...

OK. So the C++ compiler adds lib/gcc/x86_64-apple-darwin21/15.0.0/../../../../include/c++/15.0.0 etc. which correctly locates the C++ includes.

The Ada compiler uses its own includes which are installed in: $prefix/lib/gcc/x86_64-apple-darwin21/15.0.0/ada{include, lib}

So my question is now "what are you expecting to find in $prefix/include that means it should be added implicitly by the compiler"?

FWIW: I do not see clang adding $prefix/include to its include search paths either ... (it adds <sysroot>/usr/include )

[iains]

.. but asking the user to provide --sysroot= does not solve the problem

The problem that my --with-specs line addresses is that I don't know whether users will have XC or CLT installed. Asking them to either install the SDK I built against, or set SDKROOT to their SDK/use --sysroot, is going to be off-putting; and I really want to minimise the grief that potential Ada users face. It needs to Just Work.

Yeah, I totally get that - I also want the compiler to "Just Work" ... and the specs line will do that w.r.t the XC/CLT - but I think that does mean it will override an SDKROOT; have to see if there's some way around that.

[simonjwright]

OK. So the C++ compiler adds lib/gcc/x86_64-apple-darwin21/15.0.0/../../../../include/c++/15.0.0 etc. which correctly locates the C++ includes.

which is $prefix/include/c++/p.q.r. Seems a bit like a workround, but not worth fighting over.

The Ada compiler uses its own includes which are installed in: $prefix/lib/gcc/x86_64-apple-darwin21/15.0.0/ada{include, lib}

So my question is now "what are you expecting to find in $prefix/include that means it should be added implicitly by the compiler"?

Historically, AdaCore provided a whole suite of tools and utilities installed under $prefix. Ada tools (particularly the non-GCC gprbuild) know how to navigate this.

Several of those utilities need access to gmp, and my way round this was to build gmp (from the "prerequisite" source) and install it under $prefix, so that gmp.h and gmpxx.h end up in $prefix/include. That was what led me to posting this issue, because of course I couldn't find them (with a more normal config setup).

I see that Homebrew’s gcc doesn’t look in $prefix/include either, so I think I’ll need to look for a more mainstream solution!

[iains]

OK. So the C++ compiler adds lib/gcc/x86_64-apple-darwin21/15.0.0/../../../../include/c++/15.0.0 etc. which correctly locates the C++ includes.

which is $prefix/include/c++/p.q.r. Seems a bit like a workround, but not worth fighting over.

done that way so that the toolchain is relocatable when built - things are found relative to something you know.

So my question is now "what are you expecting to find in $prefix/include that means it should be added implicitly by the compiler"?

Historically, AdaCore provided a whole suite of tools and utilities installed under $prefix. Ada tools (particularly the non-GCC gprbuild) know how to navigate this.

Several of those utilities need access to gmp, and my way round this was to build gmp (from the "prerequisite" source) and install it under $prefix, so that gmp.h and gmpxx.h end up in $prefix/include. That was what led me to posting this issue, because of course I couldn't find them (with a more normal config setup).

I see that Homebrew’s gcc doesn’t look in $prefix/include either, so I think I’ll need to look for a more mainstream solution! (neither do clang-based toolchains)

In general, the compiler should not add paths that are not needed for its actions - you cannot assume (in general) that the compiler is installed in a common prefix - in fact things like Homebrew and macports (and original Xcode) have several co-installed GCC versions with ways to switch.

Adding unexpected paths to the compiler actually ends up causing great confusion and sometimes "leaks" of unintended headers and/or libraries into projects.

Probably, what you need is a mechanism to add the specific path you want for GMP et. al. .. a la --with-gmp= kind of thing

[simonjwright]

OK. So the C++ compiler adds lib/gcc/x86_64-apple-darwin21/15.0.0/../../../../include/c++/15.0.0 etc. which correctly locates the C++ includes.

which is $prefix/include/c++/p.q.r. Seems a bit like a workround, but not worth fighting over.

done that way so that the toolchain is relocatable when built - things are found relative to something you know.

If it can add lib/gcc/x86_64-apple-darwin21/15.0.0/../../../../include/c++/15.0.0 then it could just as well have added include/c++/15.0.0.

Anyway, what I need is for libgmp.dylib to be in lib/ so that it forms part of the toolchain that's relocatable when built. Don't care (much) about the headers after that.

Probably, what you need is a mechanism to add the specific path you want for GMP et. al. .. a la --with-gmp= kind of thing

I can do this by adding

export C_INCLUDE_PATH=$PREFIX/include
export CPLUS_INCLUDE_PATH=$PREFIX/include

to the build scripts of the components that need gmp.

[iains]

it could just as well have added include/c++/15.0.0.

I think the point is that "it" (the driver) does not know where include is .. only how to find it relative to some other directory. I guess it could call realpath on the output to make it a bit more readable, but the underlying operation would be the same.

====

OK.. so you have a solution ?

NOTE that none of this is anything specific to these branches or even Darwin - this handling is done in the generic driver code. If you would like a change to that - then best to file an upstream Bugzilla requesting an enhancement.

[simonjwright]

OK, I think this issue is sorted. Thanks for the help.