Closed ilg-ul closed 4 months ago
In case the folder structure of my distribution is not clear, here are the first 2 levels of the build result:
% tree -L 2 /Users/ilg/Work/xpack-dev-tools-build/gcc-14.0.1-1/darwin-x64/application
/Users/ilg/Work/xpack-dev-tools-build/gcc-14.0.1-1/darwin-x64/application
├── README.md
├── bin
│ ├── c++
│ ├── cpp
│ ├── g++
│ ├── gcc
│ ├── gcov
│ ├── gcov-dump
│ ├── gcov-tool
│ ├── gdb
│ ├── gdb-add-index
│ ├── gfortran
│ └── lto-dump
├── distro-info
│ └── licenses
├── include
│ ├── c++
│ └── gdb
├── lib
│ ├── gcc
│ ├── libatomic.1.dylib
│ ├── libatomic.a
│ ├── libatomic.dylib -> libatomic.1.dylib
│ ├── libatomic.la
│ ├── libcc1.0.so
│ ├── libcc1.a
│ ├── libcc1.la
│ ├── libcc1.so -> libcc1.0.so
│ ├── libgcc_s.1.1.dylib
│ ├── libgcc_s.1.dylib
│ ├── libgfortran.5.dylib
│ ├── libgfortran.a
│ ├── libgfortran.dylib -> libgfortran.5.dylib
│ ├── libgfortran.la
│ ├── libgfortran.spec
│ ├── libgomp.1.dylib
│ ├── libgomp.a
│ ├── libgomp.dylib -> libgomp.1.dylib
│ ├── libgomp.la
│ ├── libgomp.spec
│ ├── libitm.1.dylib
│ ├── libitm.a
│ ├── libitm.dylib -> libitm.1.dylib
│ ├── libitm.la
│ ├── libitm.spec
│ ├── libquadmath.0.dylib
│ ├── libquadmath.a
│ ├── libquadmath.dylib -> libquadmath.0.dylib
│ ├── libquadmath.la
│ ├── libssp.0.dylib
│ ├── libssp.a
│ ├── libssp.dylib -> libssp.0.dylib
│ ├── libssp.la
│ ├── libssp_nonshared.a
│ ├── libssp_nonshared.la
│ ├── libstdc++.6.dylib
│ ├── libstdc++.6.dylib-gdb.py
│ ├── libstdc++.a
│ ├── libstdc++.dylib -> libstdc++.6.dylib
│ ├── libstdc++.la
│ ├── libstdc++exp.a
│ ├── libstdc++exp.la
│ ├── libstdc++fs.a
│ ├── libstdc++fs.la
│ ├── libsupc++.a
│ └── libsupc++.la
├── libexec
│ ├── gcc
│ ├── libc++.1.0.dylib
│ ├── libc++.1.dylib -> libc++.1.0.dylib
│ ├── libc++abi.1.0.dylib
│ ├── libexpat.1.9.2.dylib
│ ├── libexpat.1.dylib -> libexpat.1.9.2.dylib
│ ├── libgmp.10.dylib
│ ├── libiconv.2.dylib
│ ├── libisl.23.dylib
│ ├── libmpc.3.dylib
│ ├── libmpfr.6.dylib
│ ├── libncursesw.6.dylib
│ ├── libz.1.3.1.dylib
│ ├── libz.1.dylib -> libz.1.3.1.dylib
│ └── libzstd.1.5.5.dylib
└── share
├── gcc-14.0.1
└── gdb
14 directories, 72 files
It is somewhat unusual to have libs installed in libexec .. you will likely need to add -rpaths to the libraries there "../lib" and to executables in libexec/gcc that use those libraries so that they can locate them ..
Mixing absolute and @rpath libraries should also work - providing that the absolute path libraries are actually installed in their expected position.
I am not sure exactly what home-brew is doing - but I think it builds with the @rpaths to avoid the configuration and test hassles .. and then re-writes the libraries to absolute paths using install_name_tool ... @fxcoudert ?
I strictly control the rpath of my binaries; I prefer the build to produce absolute paths which are post-processed and turned into @rpath/name.dylib
after the libraries are copied to libexec
.
The problems here are different and I don't think they are related to Homebrew.
The first problem is that while building libstdc++.6.dylib
a reference to the system libiconv is generated, although there is locally compiled version; I expect an issue in the configure script.
The second problem is that the binaries (like lto-dump
) have mixed references, most libraries use absolute references, except @rpath/libzstd.1.5.5.dylib
although all libraries have similar paths definitions; I expect an issue in the configure script.
I strictly control the rpath of my binaries; I prefer the build to produce absolute paths which are post-processed and turned into
@rpath/name.dylib
after the libraries are copied tolibexec
.
OK - but the clients of those libraries will still need an -rpath ....
to allow them to be found, right?
The problems here are different and I don't think they are related to Homebrew.
The first problem is that while building
libstdc++.6.dylib
a reference to the system libiconv is generated, although there is locally compiled version; I expect an issue in the configure script.
Hmm I'd expect that it's more to do with the fact you'd need to point to the libraries you intend to use during the configure/build .. the configure script has no knowledge of special installations - I'd think you need to add some LDFLAGS{_FOR_TARGET} to the configure and build.
The second problem is that the binaries (like
lto-dump
) have mixed references, most libraries use absolute references, except@rpath/libzstd.1.5.5.dylib
although all libraries have similar paths definitions; I expect an issue in the configure script.
The libzstd build is outside of the GCC one though?
You are passing --with-libiconv-prefix=
and it is used in 3 configure files: gcc
, libcpp
and libstdc++-v3
. You should be able to check in all three places, in config.log
, what happened and why the system library was linked instead of the one specified.
I did a run on a M1 macOS 11.6, this time using the Apple clang and the results are the same:
/usr/lib/libiconv.2.dylib
referred by libstdc++.6.dylib
@rpath/libzstd.1.dylib
referred by lto-dump
So the Arm binaries seem ok.
OK.. so as @fxcoudert says, you need to look though the config logs to find out why --with-libiconv-prefix=
is being ignored (I am assuming that your build script makes sure to build/install the dependencies first).
I am getting the impression that your installation might also have dependencies on the home-brew installation it's using?
To deal with this as a GCC bug, we need to reproduce the issue without all of the external interactions (i.e. to find what the underlying issue is).
I am getting the impression that your installation might also have dependencies on the home-brew installation it's using?
You mean lld
? If so, it was used only during tests on older macOS-es.
you need to look though the config logs to find out why --with-libiconv-prefix= is being ignored
I'll try to do this.
note that, if ld cannot find a library in a -L
provided at link-time (e.g. it is not yet installed) but it exists in /usr/lib
(which is the case for libintl) then it will use the one in /usr/lib
. Arguably the silent fallback to /usr/lib
is a misfeature - but it's been there "forever" and is not likely to be removed.
If you link against libzstd when it's name is "@rpath/libzstd..." then that will be in the executables - even if you subsequently re-write the name of libzstd - you will need to iterate over the clients and rewrite the dependency names.
If you link against libzstd when it's name is "@rpath/libzstd..." then that will be in the executables
Good hint! 👍
The zstd build insisted on producing a library with the @rpath/libzstd.1.dylib
name, apparently regardless my configuration options. I ended up patching the code and now the resulting name is the absolute path, like for all other libraries, and GCC treats them all consistently.
So one problem solved, not a GCC issue. :-)
It remains to investigate why the path to the libraries is ignored when building libstdc++.6.dylib
. I added a set -x
at the beginning of the libstdc++-v3/configure
script and I'll try to understand where the logic fails; hopefully by tomorrow I'll know more.
If you have any suggestions how to better diagnose this, please let me know.
If you have any suggestions how to better diagnose this, please let me know.
First look at the config.log in <arch>-apple-darwinNN/libstdc++-v3
and see it you can see why the --with-libiconv-prefix=
is being ignored (or not passed)
see why the --with-libiconv-prefix= is being ignored (or not passed)
Oops! The classical ones (--with-gmp
, --with-isl
, --with-mpc
, --with-mpfr
) are passed, but --with-libiconv-prefix
is not!
In the top Makefile
, I see that --with-libiconv-prefix
is present only in TOPLEVEL_CONFIGURE_ARGUMENTS
and HOST_CONFIGARGS
, and not in BUILD_CONFIGARGS
or TARGET_CONFIGARGS
If I can read the Makefile right, the 3rd stage libstdc++-v3/configure
is called at line 15579 with $(TARGET_CONFIGARGS)
, which explains why the libiconv path is not there.
Apparently I can pass additional STAGE3_CONFIGURE_FLAGS
.
What would be the best approach?
You need to add this to the target configuration information - --with-libiconv-prefix is for the host - seeconfigure -- help
... target_configargs additional configure arguments for target directories
You need to add this to the target configuration
Right, thank you.
... target_configargs
This variable is a bit too obscure for me, I took a shortcut and used LDFLAGS_FOR_TARGET=
to directly pass the path.
Plus that I did not like libstdc++
to reference libexec/libiconv.2.dylib
, since this is a host library; thus I compiled a static libiconv
in a separate folder and passed that path to libstdc++
.
I think that the result is cleaner now:
ilg@wksi ~ % objdump --macho --dylibs-used /Users/ilg/Work/xpack-dev-tools-build/gcc-14.0.1-1/darwin-x64/application/lib/libstdc++.6.dylib
/Users/ilg/Work/xpack-dev-tools-build/gcc-14.0.1-1/darwin-x64/application/lib/libstdc++.6.dylib:
/Users/ilg/Work/xpack-dev-tools-build/gcc-14.0.1-1/darwin-x64/application/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.33.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1336.61.1)
Since you pointed up the distinction between host and target libraries, I checked the folder where I expect to have the target libraries and I have these (see the gcc/x86_64-apple-darwin23.3.0
folder):
ilg@wksi ~ % find /Users/ilg/Work/xpack-dev-tools-build/gcc-14.0.1-1/darwin-x64/application/lib -name '*.dylib' -o -name '*.so'
/Users/ilg/Work/xpack-dev-tools-build/gcc-14.0.1-1/darwin-x64/application/lib/libgfortran.dylib
/Users/ilg/Work/xpack-dev-tools-build/gcc-14.0.1-1/darwin-x64/application/lib/libssp.0.dylib
/Users/ilg/Work/xpack-dev-tools-build/gcc-14.0.1-1/darwin-x64/application/lib/libssp.dylib
/Users/ilg/Work/xpack-dev-tools-build/gcc-14.0.1-1/darwin-x64/application/lib/libstdc++.6.dylib
/Users/ilg/Work/xpack-dev-tools-build/gcc-14.0.1-1/darwin-x64/application/lib/libatomic.dylib
/Users/ilg/Work/xpack-dev-tools-build/gcc-14.0.1-1/darwin-x64/application/lib/libgomp.dylib
/Users/ilg/Work/xpack-dev-tools-build/gcc-14.0.1-1/darwin-x64/application/lib/libatomic.1.dylib
/Users/ilg/Work/xpack-dev-tools-build/gcc-14.0.1-1/darwin-x64/application/lib/gcc/x86_64-apple-darwin23.3.0/14.0.1/plugin/libcc1plugin.0.so
/Users/ilg/Work/xpack-dev-tools-build/gcc-14.0.1-1/darwin-x64/application/lib/gcc/x86_64-apple-darwin23.3.0/14.0.1/plugin/libcp1plugin.0.so
/Users/ilg/Work/xpack-dev-tools-build/gcc-14.0.1-1/darwin-x64/application/lib/gcc/x86_64-apple-darwin23.3.0/14.0.1/plugin/libcc1plugin.so
/Users/ilg/Work/xpack-dev-tools-build/gcc-14.0.1-1/darwin-x64/application/lib/gcc/x86_64-apple-darwin23.3.0/14.0.1/plugin/libcp1plugin.so
/Users/ilg/Work/xpack-dev-tools-build/gcc-14.0.1-1/darwin-x64/application/lib/libitm.dylib
/Users/ilg/Work/xpack-dev-tools-build/gcc-14.0.1-1/darwin-x64/application/lib/libitm.1.dylib
/Users/ilg/Work/xpack-dev-tools-build/gcc-14.0.1-1/darwin-x64/application/lib/libgfortran.5.dylib
/Users/ilg/Work/xpack-dev-tools-build/gcc-14.0.1-1/darwin-x64/application/lib/libquadmath.0.dylib
/Users/ilg/Work/xpack-dev-tools-build/gcc-14.0.1-1/darwin-x64/application/lib/libcc1.0.so
/Users/ilg/Work/xpack-dev-tools-build/gcc-14.0.1-1/darwin-x64/application/lib/libgcc_s.1.1.dylib
/Users/ilg/Work/xpack-dev-tools-build/gcc-14.0.1-1/darwin-x64/application/lib/libcc1.so
/Users/ilg/Work/xpack-dev-tools-build/gcc-14.0.1-1/darwin-x64/application/lib/libstdc++.dylib
/Users/ilg/Work/xpack-dev-tools-build/gcc-14.0.1-1/darwin-x64/application/lib/libgcc_s.1.dylib
/Users/ilg/Work/xpack-dev-tools-build/gcc-14.0.1-1/darwin-x64/application/lib/libgomp.1.dylib
/Users/ilg/Work/xpack-dev-tools-build/gcc-14.0.1-1/darwin-x64/application/lib/libquadmath.dylib
Are the libc?1plugin
shared libraries really target libraries? Who uses them?
Since you pointed up the distinction between host and target libraries, I checked the folder where I expect to have the target libraries and I have these (see the
gcc/x86_64-apple-darwin23.3.0
folder):
looks right
Are the
libc?1plugin
shared libraries really target libraries? Who uses them?
AFAIU - they are plugins for libcc1 which is used by GDB (so, yes, for supporting target debugging)
I do occasionally build GDB for x86_64, where there is some support (not yet for Arm64).
they are plugins for libcc1 which is used by GDB (so, yes, for supporting target debugging)
Well, IMHO GDB is a host binary, so these plugins are somehow similar to the LTO plugins, thus host related, but my opinion probably is not relevant.
What might be relevant, is that the target libraries in the lib
folder should have no references to the host libraries in libexec
.
What do you think about this?
You are doing a native build so host==target and the separation you are looking for is not present
(your layout is non-standard and special - so you will need to deal with doing everything that the normal GNU layout does as a separate job).
Well, IMHO GDB is a host binary, so these plugins are somehow similar to the LTO plugins, thus host related, but my opinion probably is not relevant.
Yes GDB is a host binary, but imagine if you were debugging (via some remote connection) a cross-complied binary - then you might well need per-target plugins.
lib/
(target ones too when host==target)<target-triple>/{lib, bin,...}
your layout is non-standard and special - so you will need to deal with doing everything that the normal GNU layout does as a separate job
My layout is special because the distributed binaries must be standalone/self-contained in order to run on any macOS >= 10.13, regardless what shared libraries are present in the system.
AFAICT, GCC alone (or clang) cannot generate such distributions, thus the need for some post-processing to bring all libraries in libexec
and make everything self-contained.
Things are not trivial, but my scripts to achieve this are now quite functional and most tools I need are already available in this format (https://github.com/xpack-dev-tools/).
To conclude, GCC has separate configurations for each build step (host, build, target).
Some of the configuration options, like --with-gmp/mpfr/mpc/isl
are documented as applicable to the host platform only. (https://gcc.gnu.org/install/configure.html), however they are also passed when building target related libraries, like libstdc++
. On the other hand, --with-libiconv-prefix
, which would be needed in building libstdc++
, is not passed, and the documentation does not mention it being host only.
I think that a clear grouping of these options (host/build/target), fully documented, would be useful, and would help decide in the code which options go to which configure steps.
However this is probably beyond the scope of this project, and must be addressed in the upstream project.
As a workaround, there are several configuration options (like --with-stage1-ldflags
, --with-toot-ldflags
), and environment variables (like LDFLAGS_FOR_TARGET
) that can be used to pass additional settings to various steps. The documentation is not great, and some experimenting is required to set them correctly.
However this is probably beyond the scope of this project, and must be addressed in the upstream project.
Indeed, this is nothing specific to the Arm64 branch for Darwin (or even to Darwin) - it should be raised on GCC's Bugzilla either as a bug/enhancement request (it will not get any attention here except from me); There is some discussion about whether libstdc++ should use the target libiconv (or whether it should be possible to build it in-tree too).
I did a build with the trunk, bootstrapped with Homebrew llvm 17, on an Intel macOS 14.3.1 & CLT 15.1 (I had issues with 15.3, and did not upgrade yet)
The GCC build completed, but the post processing validations failed with an unexpected library:
The reason is that in my distribution I also compile libiconv from sources, and the expectation is that the local library will be used.
This happen correctly for other references, for example:
To be noted that there is also an inconsistency with
libzstd
, which is linked with relative rpath, not the absolute path like all other libraries.After all the libraries referred by the compiler binaries are copied to
libexec
, the executable located inbin
is post-processed to:The configuration includes identical definitions for all libraries (all compiled locally from sources), so the expectation is that all are processed the same:
I know that my builds are a bit more complex than usual; that's because the goal is to produce binaries that include all required libraries and can be executed from any folder; thus the need for rpath post processing, to refer to the libraries collected in libexec.
If you think that I did something wrong in my build, please let me know, and I'll retest.