gcc10 not building on Tiger due to "unwind.h" not being found during build.

[kencu]

Discussion moved from:

https://github.com/macports/macports-ports/pull/13995

[kencu]

As a data point, I built gcc11 on Tiger in January, using gcc7 from MacPorts to build it, without any patches from the current gcc11 release. I did not see this unwind.h issue there.

I built it referencing the newer ld64-97 and cctools 949 currently used in MacPorts for Tiger.

[barracuda156]

@kencu Did you build for ppc, i386 or universal?

[iains]

yeah, there are known issues with

• Xcode-2.5 cctools (as does not produce '1' output when it fails to parse command line items
• and I think ld64-62 has some problems with newer C++ stuff.

I have generally been using the tools from Xcode 3.1.4 as the bootstrap ones on Tiger.

ld64-97 I suspect is probably the best of the versions without pending patches.

cctools 949 would definitely need patches for ppc64, I have not actually checked it on ppc (but I think we can leave ppc64 that to one side on Tiger for now, since there's only very limited system support for ppc64). Again, academic if the macports default is --disable-multilib.

have to be careful that the newer tools do not generate load commands that darwin8 does not understand ... (one reason for me choosing the minimum set that worked, rather than the newest available).

[iains]

plus: gcc-4.0 does not build 10.3 - but gcc-4.2 should be able to.

[kencu]

@kencu Did you build for ppc, i386 or universal?

ppc.

[iains]

given that the Xcode-2.5 toolchain does not build gcc-10.3

the question I'd pose is - "what is the intermediate bootstrap toolchain that would be best here"?

I realise it is a pain for Darwin8 to have to make an extra step - but that, I think, is a penalty we have to accept in supporting the system.

(FWIW: we could, probably build on Darwin 7 - but then we'd have to start out by building a lot of tools - from make onwards, in the end I punted on doing this - even though I do have darwin 7 on an old [2002] tibook ... it is easier to cross-compile for it on a newer system).

[kencu]

I would think that MacPorts base is the defacto floor out there. It builds the following from system roots at present:

apple-gcc42, cctools 949, ld64-97, a newer gmake

Those tools will then build (eventually, hopefully) gcc10-bootstrap, and then things flow from there along the standard MacPorts pathways.

[iains]

which make?

I'd say that toolset should work to build cmake <= 3.9.6 and gcc-10.3, so then you'd have the basis for a c++11 step.

I'll try to fit in building that set to use as a pre-bootstrap .... dunno when tho, exactly.

[kencu]

In the gcc10-bootstrap port, I integrated a build of gmake 3.8.1. (well, I did it in gcc7-bootstrap, which became gcc10-bootstrap).

The current gmake in Macports is 4.3. To be honest, that is too new sometimes, and builds fail.

I believe all MacOS systems are tied to gmake 3.8.1 forever due to GPL3, so macports should provide that for Tiger to use for all it's builds, I think. Easy addition to do if needed.

[barracuda156]

I would think that MacPorts base is the defacto floor out there. It builds the following from system roots at present:

apple-gcc42, cctools 949, ld64-97, a newer gmake

Those tools will then build (eventually, hopefully) gcc10-bootstrap, and then things flow from there along the standard MacPorts pathways.

If it happens to be easier, we also may use macports-gcc-7 to build gcc10 on Tiger, but that is an extra step, which is painful on most G4 systems.

[kencu]

The basic MacPorts toolset on Tiger (apple-gcc42, etc) can build cmake 3.9.6 and that I believe is the last one. I made that "cmake-bootstrap". I never tried building it with the system tools, though.

Once gcc10-bootstrap works, we can make "cmake-bootstrap" the current cmake I think, as gcc10 will likely be able to build cmake for many years going forward.

[iains]

It is not possible to avoid the extra step; the system tools are too old to go straight to 10.x.

The only question is "which is the extra step set" - AFAICT the one proposed by @kencu seems reasonable (and not too too much extra to build).

I suppose you could abandon 10.x as the "stepping stone" on darwin8, but then that would make it more different from later systems ..

[kencu]

it would be best to go from apple-gcc42 to gcc10-bootstrap if we can, as all other systems will be using it to bootstrap.

[evanmiller]

Original issue (ahem) still manifests with the libunwind-headers port deactivated.

Could this have anything to do with it?

https://github.com/gcc-mirror/gcc/blob/16e2427f50c208dfe07d07f18009969502c25dc8/libgcc/config.host#L1133-L1141

Looks like some kind of Tiger special-casing having to do with unwind headers.

Let me know if there is specific debug or log information that would be helpful for debugging.

[kencu]

Evan, right about here I usually try to build outside Macports, just in a folder. It's easy.

If that works, and MacPorts portfile doesn't, at least you know what kind of problem you have.

as mentioned, gcc11 builds on Tiger.

[iains]

Original issue (ahem) still manifests with the libunwind-headers port deactivated.

Could this have anything to do with it?

https://github.com/gcc-mirror/gcc/blob/16e2427f50c208dfe07d07f18009969502c25dc8/libgcc/config.host#L1133-L1141

Looks like some kind of Tiger special-casing having to do with unwind headers.

Tiger has a fallback unwind frame handler - it is the last system to do that (because after that the system libraries have dwarf unwind) - that is the special casing.

It should not affect the unwind_header configure item that is what you need to check - it is referenced in the Makefile as the port-specific file copied into unwind.h

[kencu]

Here's how I built gcc11 on Tiger, using the existing gcc7 in MacPorts and the standard cctools and ld64-97 installed by MacPorts. I believe I deactivated zstd first to allow it to build, but you can just turn zstd off with the configure arg.

I would hope that gcc10 could build with the apple-gcc42 compilers specified, but I have not tried it.

$ ./gcc -v
Using built-in specs.
COLLECT_GCC=./gcc
COLLECT_LTO_WRAPPER=/opt/gcc/gcc11/libexec/gcc/powerpc-apple-darwin8/11.2.0/lto-wrapper
Target: powerpc-apple-darwin8
Configured with: ../configure --prefix=/opt/gcc/gcc11 --build=powerpc-apple-darwin8 --with-as=/opt/local/bin/as --with-ld=/opt/local/bin/ld --enable-languages=all CC=/opt/local/bin/gcc-mp-7 CXX=/opt/local/bin/g++-mp-7
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 11.2.0 (GCC) 

NB. I did it the "old" way, with the build folder inside the gcc folder. That is frowned on these days, you are supposed to use a build folder beside the gcc folder instead.

[kencu]

notice I did not force dwarf2 with any special configure arg here. That is still rather voodoo, and I have not as yet fully sorted out when I will or will not get that boostrap comparison failure business. If I do get it, I then try the dwarf2-forcing arg.

[evanmiller]

unwind_header is correctly set to unwind-generic.h in the configure step.

Manually running /path/to/bins/make unwind.h in the libgcc build folder creates a symlink to unwind-generic.h.

However, the unwind.h symlink step is not appearing in the build log. This leads me to suspect that make is not computing unwind.h as a target correctly. Perhaps the culprit is the make 3.81 that is being downloaded specially for Tiger in the Portfile?

[evanmiller]

Well, a plain make in the libgcc build folder isn't triggering the unwind.h target with make 4.3 either. But after running make unwind.h manually, the compilation is now proceeding.

[kencu]

Tiger's gmake 3.80 is too old to build gcc.

Every MacOS system after Tiger has gmake 3.81 installed, so that's why I used gmake 3.81, and it worked to build gcc11.

[kencu]

If I get up some gumption I'll try building gcc10 outside of MacPorts on my Tiger box using apple-gcc42 to build it. Perhaps tonight.

[evanmiller]

...then it fails with:

:info:build /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_gcc10-bootstrap/gcc10-bootstrap/work/build/./gcc/xgcc -B/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_gcc10-bootstrap/gcc10-bootstrap/work/build/./gcc/ -B/opt/local/libexec/gcc10-bootstrap/powerpc-apple-darwin8.11.0/bin/ -B/opt/local/libexec/gcc10-bootstrap/powerpc-apple-darwin8.11.0/lib/ -isystem /opt/local/libexec/gcc10-bootstrap/powerpc-apple-darwin8.11.0/include -isystem /opt/local/libexec/gcc10-bootstrap/powerpc-apple-darwin8.11.0/sys-include   -fno-checking -O2  -g -O2 -pipe -isysroot/Developer/SDKs/MacOSX10.4u.sdk -arch ppc -DIN_GCC    -W -Wall -Wwrite-strings -Wcast-qual -Wno-format -Wstrict-prototypes -Wmissing-prototypes -Wold-style-definition  -isystem ./include   -mmacosx-version-min=10.4 -Wa,-force_cpusubtype_ALL -fno-common -mlong-double-128 -g -DIN_LIBGCC2 -fbuilding-libgcc -fno-stack-protector   -dynamiclib -nodefaultlibs \
:info:build     -install_name /opt/local/libexec/gcc10-bootstrap/lib/libgcc_ehs.1.1.dylib \
:info:build     -o ./libgcc_ehs.dylib -compatibility_version 1 -current_version 1.1 \
:info:build     unwind-dw2_s.o unwind-dw2-fde-darwin_s.o unwind-c_s.o darwin-world_s.o -lc
:info:build Undefined symbols:
:info:build   "__Unwind_fallback_frame_state_for", referenced from:
:info:build       _uw_frame_state_for in unwind-dw2_s.o
:info:build ld: symbol(s) not found
:info:build collect2: error: ld returned 1 exit status
:info:build make[3]: *** [libgcc_ehs.dylib] Error 1

[kencu]

I have gcc10 building away on Tiger now. It's still running, but I seem to be making progress past the unwind.h issue at least:

$ uname -a
Darwin pmg5.local 8.11.0 Darwin Kernel Version 8.11.0: Wed Oct 10 18:26:00 PDT 2007; root:xnu-792.24.17~1/RELEASE_PPC Power Macintosh powerpc
pmg5:/Users/Shared/gcc/buildgcc user$ find . | grep unwind
./gcc/include/unwind.h
./powerpc-apple-darwin8/libgcc/md-unwind-support.h
./powerpc-apple-darwin8/libgcc/unwind-c_s.dep
./powerpc-apple-darwin8/libgcc/unwind-c_s.o
./powerpc-apple-darwin8/libgcc/unwind-dw2-fde-darwin_s.dep
./powerpc-apple-darwin8/libgcc/unwind-dw2-fde-darwin_s.o
./powerpc-apple-darwin8/libgcc/unwind-dw2_s.dep
./powerpc-apple-darwin8/libgcc/unwind-dw2_s.o
./powerpc-apple-darwin8/libgcc/unwind-sjlj_s.dep
./powerpc-apple-darwin8/libgcc/unwind-sjlj_s.o
./powerpc-apple-darwin8/libgcc/unwind.h
./powerpc-apple-darwin8/ppc64/libgcc/md-unwind-support.h
./powerpc-apple-darwin8/ppc64/libgcc/unwind-c.dep
./powerpc-apple-darwin8/ppc64/libgcc/unwind-c.o
./powerpc-apple-darwin8/ppc64/libgcc/unwind-c_s.dep
./powerpc-apple-darwin8/ppc64/libgcc/unwind-c_s.o
./powerpc-apple-darwin8/ppc64/libgcc/unwind-dw2-fde-darwin.dep
./powerpc-apple-darwin8/ppc64/libgcc/unwind-dw2-fde-darwin.o
./powerpc-apple-darwin8/ppc64/libgcc/unwind-dw2-fde-darwin_s.dep
./powerpc-apple-darwin8/ppc64/libgcc/unwind-dw2-fde-darwin_s.o
./powerpc-apple-darwin8/ppc64/libgcc/unwind-dw2.dep
./powerpc-apple-darwin8/ppc64/libgcc/unwind-dw2.o
./powerpc-apple-darwin8/ppc64/libgcc/unwind-sjlj.dep
./powerpc-apple-darwin8/ppc64/libgcc/unwind-sjlj.o
./powerpc-apple-darwin8/ppc64/libgcc/unwind-sjlj_s.dep
./powerpc-apple-darwin8/ppc64/libgcc/unwind-sjlj_s.o
./powerpc-apple-darwin8/ppc64/libgcc/unwind.h

I'm not building it with a Portfile, but I am using apple-gcc42 as the boot compiler:

../gcc-10.3.0/configure --prefix=/opt/gcc/gcc10 --build=powerpc-apple-darwin8 --without-zstd --enable-languages=all CC=/opt/local/bin/gcc-apple-4.2 CXX=/opt/local/bin/g++-apple-4.2 > config.txt && time nice make -j4 >b.txt 2>e.txt

I'll let you know when it's done, if it finishes.

[kencu]

Oh, I built gmake 3.81 first and set up the path to use that.

[iains]

...then it fails with:

:info:build /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_gcc10-bootstrap/gcc10-bootstrap/work/build/./gcc/xgcc -B/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_gcc10-bootstrap/gcc10-bootstrap/work/build/./gcc/ -B/opt/local/libexec/gcc10-bootstrap/powerpc-apple-darwin8.11.0/bin/ -B/opt/local/libexec/gcc10-bootstrap/powerpc-apple-darwin8.11.0/lib/ -isystem /opt/local/libexec/gcc10-bootstrap/powerpc-apple-darwin8.11.0/include -isystem /opt/local/libexec/gcc10-bootstrap/powerpc-apple-darwin8.11.0/sys-include   -fno-checking -O2  -g -O2 -pipe -isysroot/Developer/SDKs/MacOSX10.4u.sdk -arch ppc -DIN_GCC    -W -Wall -Wwrite-strings -Wcast-qual -Wno-format -Wstrict-prototypes -Wmissing-prototypes -Wold-style-definition  -isystem ./include   -mmacosx-version-min=10.4 -Wa,-force_cpusubtype_ALL -fno-common -mlong-double-128 -g -DIN_LIBGCC2 -fbuilding-libgcc -fno-stack-protector   -dynamiclib -nodefaultlibs \
:info:build     -install_name /opt/local/libexec/gcc10-bootstrap/lib/libgcc_ehs.1.1.dylib \
:info:build     -o ./libgcc_ehs.dylib -compatibility_version 1 -current_version 1.1 \
:info:build     unwind-dw2_s.o unwind-dw2-fde-darwin_s.o unwind-c_s.o darwin-world_s.o -lc
:info:build Undefined symbols:
:info:build   "__Unwind_fallback_frame_state_for", referenced from:
:info:build       _uw_frame_state_for in unwind-dw2_s.o
:info:build ld: symbol(s) not found
:info:build collect2: error: ld returned 1 exit status
:info:build make[3]: *** [libgcc_ehs.dylib] Error 1

I think this is a real bug, with the unwind fallback code not being included in the new EHS library, so that if you built 10.3 without the backports it would be OK - but master and 10 or 11 with the libgcc_s changes backported will fail on powerpc-darwin8 (only, because it's the only Darwin platform still using the fallback code), it should be OK on i686-darwin8.

When I have a few spare ns, will try to fix this .. let me check if we even need the fallback code on 10.4.11 ... I would imagine everyone would be on the latest.

[kencu]

my unpatched stock gcc 10.3.0 is building away just fine on Tiger, 5 hours in so far. It looks like it will finish sometime during the night, so I'll report back in the morning.

[evanmiller]

I think the original bug is right here:

https://github.com/macports/macports-ports/blob/454b83de4957e3b979085c52c79a450486d0ef95/lang/gcc10-bootstrap/files/patch-iains-ppc.diff#L1246

My working theory is that this line should also contain unwind.h. Without that line, unwind.h does not exist at the moment when unwind-dw2.c is built. On 10.5+, the #include falls back to the system unwind.h and compilation proceeds. Since Tiger lacks a system unwind.h, the include fails.

The bug is difficult to see because unwind.h is generated moments later for a different part of the Makefile. (At least, according to this theory...) Being part of the backport patches, this failure is not expected to show up on Ken's stock build.

In any event I will patch the patch and see what happens.

[iains]

I think the original bug is right here:

https://github.com/macports/macports-ports/blob/454b83de4957e3b979085c52c79a450486d0ef95/lang/gcc10-bootstrap/files/patch-iains-ppc.diff#L1246

DWARF-2 unwinding is not Darwin-specific (and certainly not powerpc-darwin8-specific) - odd that it has not shown up elsewhere.

A quick scan of the Makefile.in does not show the dependency explicitly - nor does a grep for unwind-dw2.o in libgcc/config.

so...

My working theory is that this line should also contain unwind.h. Without that line, unwind.h does not exist at the moment when unwind-dw2.c is built. On 10.5+, the #include falls back to the system unwind.h and compilation proceeds. Since Tiger lacks a system unwind.h, the include fails.

... you could be right, and it's just dumb luck that most platforms have an unwind.h somewhere in the includes searches.

(I've found numerous similar problems building Linux-y stuff on Darwin, where the system doesn't have things like elf.h .. these bugs happen).

I guess it needs to be fixed "upstream" and for all platforms - not a local hack for ppc-darwin8.

[iains]

BTW. This probably (almost certainly) means we need a powerpc-darwin8 t- fragment (we cannot add the dependency on the md-unwind- header to all targets) ... and we also need to add darwin-fallback.o to the list of objects for libgcc_ehs (but for ppc darwin8 only).

[iains]

hmm - the libunwind.h dependency is there - it's just indirected:

$(libgcc-objects) $(libgcc-s-objects) $(libgcc-eh-objects) \
    $(libgcov-objects) \
    $(libunwind-objects) $(libunwind-s-objects) \
    $(EXTRA_PARTS): $(LIBGCC_LINKS) libgcc_tm.h

the objects lists are built by substituting the source names (with .o) and $(LIBGCC_LINKS) contains libunwind.h

So ...AFAICT this should be working OK - and the problem might well be a buggy make version?

(the comments about md-unwind- and darwin-fallback.o remain tho)

[iains]

is there actually a public branch that contains what you are trying to build ?

[evanmiller]

@iains This is the originating commit that catap pulled in: https://github.com/iains/gcc-10-branch/commit/97a7110d12d62b700fea4ed6a0e152f833a9d4db

gthr-default.h and md-unwind-support.h are already present in LIBGCC_LINKS – so following your logic I would think either all three headers should be in t-darwin-ehs, or none of them.

[iains]

right - probably using $(LIBGCC_LINKS) libgcc_tm.h is a bit over the top;

[evanmiller]

My previous point is that there is clearly a dependency on unwind.h (since you're seeing it generated on Leopard) but it's not actually part of unwind-dw2.c's dependency tree for whatever reason. I tried different versions of make with the same result.

@kencu If you want to follow the action try the gcc-10-3-ppc branch in this repo and see what happens...

[barracuda156]

@kencu If you want to follow the action try the gcc-10-3-ppc branch in this repo and see what happens...

@iains By the way what is the config line you normally use to build your branches on 10.4.11? I will try it locally outside of Macports.

[iains]

My previous point is that there is clearly a dependency on unwind.h (since you're seeing it generated on Leopard) but it's not actually part of unwind-dw2.c's dependency tree for whatever reason. I tried different versions of make with the same result.

I do not disagree about the dependency but I'm not sure why the main makefile is not achieving it:

all: install-unwind_h-forbuild

as well as the dependencies on the object groups.

I am not against adding the header to that list (if we cannot identify why make is somehow not honouring the deps seen); but as noted above, it needs to be done upstream too.

[iains]

@iains By the way what is the config line you normally use to build your branches on 10.4.11? I will try it locally outside of Macports.

this uses a build of cctools and ld64 from Xcode 3.1.4 as noted before .. no great mysteries otherwise.

--prefix=... --build=i686-apple-darwin8 --with-dwarf2 --disable-multilib --with-ld=/opt/iains/i686-apple-darwin8/cct-698-ld64-85-2/bin/ld --with-as=/opt/iains/i686-apple-darwin8/cct-698-ld64-85-2/bin/as --enable-languages=all --with-cpu=generic

edit: the --disable-multilib is here because the box I am building on is a core duo - it cannot run the testsuite for x86_64 (although it could, of course, build the code).

[kencu]

@kencu If you want to follow the action try the gcc-10-3-ppc branch in this repo and see what happens...

The "stock" unpatched build of gcc10 on Tiger that I started for you to prove that the current gcc10 build works continues on; haven't touched it once.

I'll report back when it is fully finished.

[kencu]

well, it built but got the dreaded "bootstrap comparison failure". That's where the "dwarf2" configure arg comes in to save the day, for sure, and how I came to add it by default for MacPorts.

Anyway, it seems pretty clear that the Tiger build issue is somewhere in Iain's new patches and that is not a surprise perhaps as Tiger is no longer on his usual build-test cycle. You and he will sort out exactly what.

Always appreciated, Iain!

For the macports gcc10-bootstrap port right now today, Tiger could consider just using an unpatched gcc 10.3.0.

And, as always, as I have told folks for some years now, building a freestanding gcc is the way forward to figure out where the problem lies.

[kencu]

I do not disagree about the dependency but I'm not sure why the main makefile is not achieving it:

Over the years, our usual finding for things like this that really should work but just don’t (in gcc and elsewhere) is race conditions

[iains]

I do not disagree about the dependency but I'm not sure why the main makefile is not achieving it:

Over the years, our usual finding for things like this that really should work but just don’t (in gcc and elsewhere) is race conditions

sure, and that was what my follow-up patch was about (although races between installing the crts and using them was the observed problem there). In this case, as above, I have no objection to adding the third header - but there are some other changes to be made for [powerpc] darwin8.

As you surmised "if it ain't tested, it's broke" some fallout from removing Tiger from my weekly tests.

.. testing ppc-d8 is particularly slow with the resources I have (so I'm not likely to introduce it into the weekly testing), however I do test before a release.

[kencu]

Your dedication to the cause is legendary.

Clearly your skill set here is leaps and bounds beyond almost everyone around this issue. Many appreciate the time you spend on this, knowing full well you have lots else to do that actually makes you an income :)

Thanks, Iain.

[evanmiller]

Just to confirm on a fresh build with catap's patches: Modifying the linked patch to include unwind.h does fix the original reported issue. Obviously it's up to @iains with how to generalize and proceed, but we should have enough here to get to the next hurdle on the MacPorts side.

[iains]

Just to confirm on a fresh build with catap's patches: Modifying the linked patch to include unwind.h does fix the original reported issue.

Does that mean the build then completes without further errors - or just past this initial blocker?

[evanmiller]

Does that mean the build then completes without further errors - or just past this initial blocker?

Just past the initial blocker. I then get the undefined symbol error for __Unwind_fallback_frame_state_for reported above.

[iains]

Does that mean the build then completes without further errors - or just past this initial blocker?

Just past the initial blocker. I then get the undefined symbol error for __Unwind_fallback_frame_state_for reported above.

Right, that's what I expected - thanks for confirming.

[iains]

as a workaround for now, in libgcc/config/rs6000/t-darwin-ehs append darwin-fallback.o to the EHS objects.

(untested of course!)

I suspect the final solution will be quite similar - most likely with an extra comment ..

[evanmiller]

as a workaround for now, in libgcc/config/rs6000/t-darwin-ehs append darwin-fallback.o to the EHS objects.

(untested of course!)

I suspect the final solution will be quite similar - most likely with an extra comment ..

Seems to do the trick! libgcc finished compiling and now it's on to the next configuration step...

[kencu]

my rebuild of the standard build of gcc 10.3.0 using --with-dwarf2 failed again on Tiger with the dreaded "bootstrap comparison failure", so I'm out of this for the time being.

Glad to see you are helping Iain get his patchset "Tiger-ready".