search

iains / gcc-darwin-arm64

GCC master branch for Darwin with experimental support for Arm64. Currently GCC-15.0.0 [September 2024]
GNU General Public License v2.0
268 stars 33 forks source link

Useful search paths omitted #93

Closed simonjwright closed 2 years ago

simonjwright commented 2 years ago

Configuring with --with-sysroot=$SDKROOT or --with-build-sysroot=$SDKROOT - as you have to nowadays - removes some search path members.

Summary

We should include /usr/local - the argument for /Library/Frameworks isn't so obvious.

Details

With clang, I see

#include <...> search starts here:
 /usr/local/include
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.1.6/include
 /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
 /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks (framework directory)
End of search list.

and

Library search paths:
    /usr/local/lib
    /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib
Framework search paths:
    /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/

With an old GCC 10.1, built on darwin15 for x86_64 without --sysroot, I see

#include <...> search starts here:
 /opt/gcc-10.1.0/bin/../lib/gcc/x86_64-apple-darwin15/10.1.0/include
 /opt/gcc-10.1.0/bin/../lib/gcc/x86_64-apple-darwin15/10.1.0/include-fixed
 /usr/local/include
 /opt/gcc-10.1.0/bin/../lib/gcc/../../include
 /System/Library/Frameworks
 /Library/Frameworks
 /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include
 /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include
End of search list.

and

Library search paths:
    /opt/gcc-10.1.0/lib/gcc/x86_64-apple-darwin15/10.1.0
    /opt/gcc-10.1.0/lib/gcc
    /opt/gcc-10.1.0/lib
    /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib
    /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib
    /usr/lib
    /usr/local/lib
Framework search paths:
    /Library/Frameworks/
    /System/Library/Frameworks/

whereas a GCC 12.1.0 x86_64-cross-aarch64 compiler (configured using --with-sysroot=$COMMAND_LINE_TOOLS) gives

GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/local/include"
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:
 /Users/simon/tmp/aarch64-apple-darwin-cross-8/lib/gcc/aarch64-apple-darwin21/12.1.0/include
 /Users/simon/tmp/aarch64-apple-darwin-cross-8/lib/gcc/aarch64-apple-darwin21/12.1.0/include-fixed
 /Users/simon/tmp/aarch64-apple-darwin-cross-8/lib/gcc/aarch64-apple-darwin21/12.1.0/../../../../aarch64-apple-darwin21/include
 /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include
 /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks
End of search list.

and

Library search paths:
    /Users/simon/tmp/aarch64-apple-darwin-cross-8/lib/gcc/aarch64-apple-darwin21/12.1.0
    /Users/simon/tmp/aarch64-apple-darwin-cross-8/aarch64-apple-darwin21/lib
    /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib
Framework search paths:
    /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/
iains commented 2 years ago

it is not a regression, as such, nor is it mysterious ;) and I think you are mistaken about clang adding those paths, IIUC it is done by xcrun.

The modern installations from Apple do not include any headers in / ... ... but the SDKs do not include /usr/local/{include, lib} or /Library/Frameworks .. so that those paths, relative to the SDK root are excluded.

If you do not have /usr/local/{include, lib} on an older system [where you are using / as the sysroot], then those paths will also be omitted from the search.

Actually, the compiler should not be adding those paths if a sysroot is in use, since it has no way to know whether the content of the paths is meaningful for the target (stuff leaking out of /usr/local might seem convenient but it is also a nuisance).

The compiler, itself should stay "honest" and not add random paths that do not exist in the nominated sysroot. If they are needed then add them with -I -L -F or, I guess, make a wrapper like xcrun.

to illustrate:

$ clang --sysroot=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk /source/test/hello.c -v -S

#include <...> search starts here:
 /System/Volumes/Data/XC/11.5/usr/lib/clang/11.0.3/include
 /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include
 /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks (framework directory)
End of search list.

(no /usr/local/include)

$ xcrun clang /source/test/hello.c -v -S
#include <...> search starts here:
 /usr/local/include
 /System/Volumes/Data/XC/11.5/usr/lib/clang/11.0.3/include
 /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include
 /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks (framework directory)
End of search list.

if you invoke GCC using xcrun... (you have to use the full name for gcc because Xcode hijacks gcc and g++ and calls clang for them).

$ xcrun x86_64-apple-darwin19-gcc /source/test/hello.c -v -S

#include <...> search starts here:
 /usr/local/include
 /opt/iains/x86_64-apple-darwin19/gcc-7-5-toolchain/lib/gcc/x86_64-apple-darwin19/7.5.0/include
 /opt/iains/x86_64-apple-darwin19/gcc-7-5-toolchain/lib/gcc/x86_64-apple-darwin19/7.5.0/include-fixed
 /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include
 /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks
End of search list.

edit: --with-build-sysroot= is not needed, and is buggy ( https://gcc.gnu.org/bugzilla/show_bug.cgi?id=79885 ), so I do not recommend its use.

ilg-ul commented 2 years ago

Sorry if this is not exactly to the point, but I also noticed similar issues while testing the xPack GCC distribution.

In this use case the binaries are not created on the target machine, but on one of my build machines, and are intended to run on user machines, which might run a different OS version.

The problem is aggravated by the fact that users may have either Xcode installed or only CLT. Thus I recommend to separately install CLT, even when Xcode is installed.

I wonder if all these details are properly addressed by the current GCC sources.

iains commented 2 years ago

I am not sure what you are asking for ... ... neither gcc nor clang (actual compiler) adds those paths (when SDKROOT or a sysroot is used, which is always on a modern system). ... it can be done by using xcrun ... it can be done by -I /usr/local/include -L /usr/local/lib -F /Library/Frameworks where that is a correct thing to do for a given case.

it is not always the right thing to do: e.g. if one is on an x86_64 machine with /usr/local/include,lib containing some library and cross-compiling for arm64 - the library in /usr/local/{lib, include} is not correct for arm64 - and that can lead to misconfigurations.

So you (or some wrapper, like xcrun) has/have to decide whether it's valid to add the paths for the local system,

edit: I do not think that this is something to be addressed by the GCC sources, I think there is an open source equivalent of xcrun somewhere.

simonjwright commented 2 years ago

I was hoping to be able to fix things so my users (who were very pleased to have an aarch64 Ada compiler!) wouldn't need to say -I /usr/local/include -L /usr/local/lib -F /Library/Frameworks -- which used to be the case only a few years ago.

Not helped by the fact that for Ada it would look like

gnatmake foo.adb -cargs -I /usr/local/include -largs -L /usr/local/lib -F /Library/Frameworks

Mind, I've not had any issues raised where the fix was to specify those switches, so I guess my users have worked round problems on their own, good for them.

iains commented 2 years ago

FAOD: It has really nothing to do with "used to work a few years ago", actually, if you build a compiler on Darwin11 without the CLT (or "unix command line support" on earlier Xcode) installed, but using the Xcode or /Developer SDK, exactly the same thing will happen. ..

Of course, most (but not all) folks used to take the short-cut of installing the CLT/unix support - which gave them the headers in /usr/include etc. and effectively made / into the sysroot - whereupon /usr/local/{include, lib} become part of the sysroot and therefore get searched (with sometimes unintended consequences for cross compilers, as noted before).

(BTW on Darwin9 and 10 quite a few OSS packages fail - as does Apple gcc-4.2 without the unix support installed - so I don't recommend it there).

You could (following Apple's example) make all of the compiler drivers into wrappers that set the extra params.

i.e. /usr/bin/clang does not invoke clang directly - it goes via the xcrun machinery ... you could do something similar .. so far, I have avoided this step since I am reluctant to incur another process launch for each of the millions of tests in the testsuite .....