Apply OS/2 patches

[dmik]

Since #1 is done, it's time to apply OS/2 patches.

We will take everything from https://github.com/psmedley/gcc/tree/gcc-4_9-branch-os2, plus Paul Smedley kindly offered us his latest patch set for his own build of GCC 9.2.0 (which should also include all from GitHub).

[dmik]

Note that applying patches is taking some time. While I use the same git cherry-pick approach I used in the old repo to transfer patches from a branch to a branch, it's still not as fast as one could think: since the difference between version 4.9.2 and version 9.2.0 is rather huge, some patches require a lot of aligning (and produce a lot of conflicts).

Other than that, thanks to cherry-pick all work will get its respective attribution.

[dmik]

We will also need to decide what to do with issues from https://github.com/psmedley/gcc/. Perhaps, we need to move the relevant ones here.

BTW, as cherry-pick preserves commits, references to https://github.com/psmedley/gcc/ (both to its commits and issues) are also preserved. This should be kept in mind for all cherry-picked commits.

[dmik]

I've applied all patches from the 4.9 branch (with some modifications where I saw need). Note that all applied patches contain SHAs from https://github.com/psmedley/gcc/ to simplify followups. Some patches (like regression fixes) were joined with the original ones and it's also mentioned using their SHAs.

Note that there are two patches that I didn't apply yet: https://github.com/psmedley/gcc/commit/13e9736b5c4f3dfa44ccd96893fee01e868fe3b6 and https://github.com/psmedley/gcc/commit/21e0dd64d12d316004faedd38125e06872dd937a as I first want to find the exact use cases that makes them necessary. And I don't think anything like that done on other platforms.

I also think that https://github.com/psmedley/gcc/commit/fef1630c35955e6352f9734b43d20758ce7508be is not optimal (though I've applied it for now) — this stuff should be fixed in https://github.com/bitwiseworks/libc instead. I will create a ticket later.

The next step is to apply Paul's 9.2.0 patches. IIRC, there shouldntt be much. Note that I'm not pushing my work yet because I'm rewriting history in order to make it more readable. I will push once it's done.

[dmik]

Paul's patches applied. I've also asked him for some clarifications. Now the build time.

I will also regenerate all configure scripts to let them pick up the OS/2 toolchain details w/o patching them manually (and w/o configuring a special build environment) — i.e. the same way we already do for all other ported software. I'm not sure, however, if we should remove pre-generated configure from the repository as we do in other projects. The reason is that there are too many of them and no script or such to generate them from scratch. May be I will add one, we will see.

BTW, a good instruction on how to maintain the configure scripts is located here: BTW, there is a good instruction https://gcc.gnu.org/wiki/Regenerating_GCC_Configuration

[dmik]

I've successfully regenerated all configure/libtool scripts and other stuff (there's pretty much I'd say). And although I noticed that gcc requires automake 1.15.1 while we have automake 1.14.1, it seems to have worked well so far. At least all (many) configure ran smoothly afterwards (We will port automake 1.15.1 if needed — this is easy in our env).

And then I've got a first build break rather quickly when building in gcc (which means that all prerequisites of it — like fixincludes, libiberty etc. — were built just fine):

make: *** No rule to make target `D:/Coding/gcc/master/gcc/config/i386/gstabs.h', needed by `s-gtype'.  Stop.

There is no such source file in GCC 9.2.0 but there is one in the old GCC (4.9.2). I wonder how it could get in to the new makefiles.

[dmik]

The above problem was fixed — a missing patch from Paul's set. There is another build failure much later. Most likely due to some patch problems as well (it's kinda hard to apply them cleanly outside a VCS). Investigating.

[dmik]

A short report. After some more fixes (to be committed still) the compiler itself was built successfully. However, I get a weird error when building libstdc++:

C:/usr/bin/sh.exe ../libtool --tag CXX   --mode=compile D:/Coding/gcc/master-build/./gcc/xgcc -shared-libgcc -BD:/Coding/gcc/master-build/./gcc -nostdinc++ -LD:/Coding/gcc/master-build/i386-pc-os2-emx/libstdc++-v3/src -LD:/Coding/gcc/master-build/i386-pc-os2-emx/libstdc++-v3/src/.libs -LD:/Coding/gcc/master-build/i386-pc-os2-emx/libstdc++-v3/libsupc++/.libs -BD:/Coding/gcc/master-install/i386-pc-os2-emx/bin/ -BD:/Coding/gcc/master-install/i386-pc-os2-emx/lib/ -isystem D:/Coding/gcc/master-install/i386-pc-os2-emx/include -isystem D:/Coding/gcc/master-install/i386-pc-os2-emx/sys-include     -ID:/Coding/gcc/master-build/i386-pc-os2-emx/libstdc++-v3/include/i386-pc-os2-emx -ID:/Coding/gcc/master-build/i386-pc-os2-emx/libstdc++-v3/include -ID:/Coding/gcc/master/libstdc++-v3/libsupc++   -std=gnu++98 -DDLL_EXPORT -DPIC -fno-implicit-templates  -Wall -Wextra -Wwrite-strings -Wcast-qual -Wabi=2  -fdiagnostics-show-location=once     -frandom-seed=compatibility-thread-c++0x.lo -g -O2 -march=pentium -mtune=pentium4  -std=gnu++11 -c D:/Coding/gcc/master/libstdc++-v3/src/c++11/compatibility-thread-c++0x.cc
libtool: compile:  D:/Coding/gcc/master-build/./gcc/xgcc -shared-libgcc -BD:/Coding/gcc/master-build/./gcc -nostdinc++ -LD:/Coding/gcc/master-build/i386-pc-os2-emx/libstdc++-v3/src -LD:/Coding/gcc/master-build/i386-pc-os2-emx/libstdc++-v3/src/.libs -LD:/Coding/gcc/master-build/i386-pc-os2-emx/libstdc++-v3/libsupc++/.libs -BD:/Coding/gcc/master-install/i386-pc-os2-emx/bin/ -BD:/Coding/gcc/master-install/i386-pc-os2-emx/lib/ -isystem D:/Coding/gcc/master-install/i386-pc-os2-emx/include -isystem D:/Coding/gcc/master-install/i386-pc-os2-emx/sys-include -ID:/Coding/gcc/master-build/i386-pc-os2-emx/libstdc++-v3/include/i386-pc-os2-emx -ID:/Coding/gcc/master-build/i386-pc-os2-emx/libstdc++-v3/include -ID:/Coding/gcc/master/libstdc++-v3/libsupc++ -std=gnu++98 -DDLL_EXPORT -DPIC -fno-implicit-templates -Wall -Wextra -Wwrite-strings -Wcast-qual -Wabi=2 -fdiagnostics-show-location=once -frandom-seed=compatibility-thread-c++0x.lo -g -O2 -march=pentium -mtune=pentium4 -std=gnu++11 -c D:/Coding/gcc/master/libstdc++-v3/src/c++11/compatibility-thread-c++0x.cc  -DDLL_EXPORT -DPIC -D_GLIBCXX_SHARED -o .libs/compatibility-thread-c++0x.o
In file included from D:/Coding/gcc/master/libstdc++-v3/include/std/future:38,
                 from D:/Coding/gcc/master/libstdc++-v3/src/c++11/compatibility-thread-c++0x.cc:30:
D:/Coding/gcc/master/libstdc++-v3/include/std/mutex: In function 'void std::call_once(std::once_flag&, _Callable&&, _Args&& ...) [with _Callable = void (std::thread::*)(); _Args = {std::thread*}]':
D:/Coding/gcc/master/libstdc++-v3/include/std/mutex:698:5: error: unsupported size for integer register
  698 |     }
      |     ^

This really looks like a break in some EMX code generation (i.e. a bunch of alignments for 9.2.0 in gcc/config/i386/*emx* files responsible for platform-specific generation).

[dmik]

After some experiments I found what triggers this failure. It's -D_GLIBCXX_SHARED. This define is used across libstdc++ to mark code that should only belong to a DLL version of the library. This code is mostly some compatibility symbols that were removed in newer versions but must be present in order for older apps linked against an older libstdc++ DLL could still work.

GCC builds from Paul Smedley only provide static libraries so it's not an issue there. This issue is also absent in our GCC builds (which do provide shared libraries) because these builds use a special version of libtool which doesn't specify -D_GLIBCXX_SHARED. Which means that some compatibility symbols have always been missing from our stdcpp6.dll and it's just pure luck that we didn't need this. It also may mean that the above compilation error has always been there and we just didn't trigger that because of the missing define in libtool (so the code in question was never compiled). At least, according to the 4.9.2 to 9.2.1 comparison, there are not many changes in the relevant symbols.

Note that a special version of libtool is a hack (by Knut?) that was used to build GCC version 3 on OS/2 back in times when we didn't have a linux toolchain (coreutils/autoconf/automake/m4 etc). Now that we have it, a proper libtool that doesn't need any manual hacking is generated during the configure process. And this libtool does what authors expect it to do.

I will try to get more details on the exact compile failure now by looking at the generated assembly. If I fail or find out it reqiures too much time to fix, I will have to sacrifice the compatibility symbols just like we did before.

[dmik]

Finally, I made a simplest example that makes both GCC 9.2.0 and our current RPM build of GCC 4.9.2 fail. Certainly, deserves a separate ticket — #3.

[dmik]

A minor build issue:

  slibdir= libsubdir= MULTIOSDIR=.
make[3]: Entering directory `D:/Coding/gcc/master-build/i386-pc-os2-emx/libgcc'
D:/Coding/gcc/master/libgcc/shared-object.mk:14: warning: overriding recipe for target `emx-ctordtor.o'
D:/Coding/gcc/master/libgcc/config/i386/t-emx:89: warning: ignoring old recipe for target `emx-ctordtor.o'
C:/usr/bin/sh.exe D:/Coding/gcc/master/libgcc/../mkinstalldirs ../.././gcc
/@unixroot/usr/bin/install.exe -c -m 644 libgcc_eh.a ../.././gcc/
chmod 644 ../.././gcc/libgcc_eh.a
ranlib ../.././gcc/libgcc_eh.a
/@unixroot/usr/bin/install.exe -c -m 644 ./gcc1.dll ../.././gcc/ && /@unixroot/usr/bin/install.exe -c -m 644 ./gcc1.a ../.././gcc/./ && /@unixroot/usr/bin/install.exe -c -m 644 ./libgcc_so_d.lib ../.././gcc/./ && /@unixroot/usr/bin/install.exe -c -m 644 ./libgcc_so_d.a ../.././gcc/./
install.exe: cannot remove '../.././gcc/gcc1.dll': Permission denied

This happens if your build fails and you restart it again after modifying some files in multi-job mode. I use a build script that adds new DLLs to BEGINLIBPATH and sets LIBPATHSTRICT=T — this is because the build process uses a freshly biult XGCC.EXE which may need a new DLL to work properly. Then, once the build process is restarted, all apps begin using a freshly-built GCC1.DLL. So it becomes locked. And if make decides it needs to be rebuilt (because some prerequisite has changed), you get it locked and INSTALL fails. Perhaps we should teach install automatically overwrite such DLLs...

[dmik]

With another small patch (to libquadmath due to a missing putcw) all GCC suite builds now. The next step is to try to install it.

[dmik]

Install also works quite fine. I get the following DLLs:

gcc1.dll
quadmat0.dll
ssp0.dll
stdc++6.dll

Note that GCC makefiles want stdc++6.dll as the libstdc++ name and libbssp is named ssp0.dll. Also, there is no libsupc++ as a DLL and seems to be no way to build it as such (meaning that the GCC team doesn't see any need in that). I wonder why @ydario did that manually for our GCC 4.x RPMs. Hope he notices this comment and replies.

Note that all these names are different from what our current RPMs provide. That's because in gcc4.spec Yuri builds libssp/libstdc++/libsupc++ DLLs by hand instead of trusting GCC makefiles — it was done so because of some makefile problems back then. I guess it was because we did not re-generate configure and makefile.in using our OS/2-tailored autotools and used a hacked version of libtool. But now all works and these names are the canonical ones. Which means that we again will have to create DLL wrappers for the old, custom-made names...

[dmik]

Another strange thing I notice is that GCC decided to fix not only LIBC headers (which it always used to do) but also one os2tk45 header (os2wdef.h). I guess this happened because it's now part of /usr/include/ (installed into the os2tk45 subdir there). The only thing fixed there surronding the wchar_t definition like that:

#ifndef __cplusplus
 typedef unsigned short wchar_t;
#endif

to avoid conflicts with its own definition. But this doesn't seem actually necessary. More over, this file belongs to a completely different package and I will try to disable that.

Other than that, it all looks good and I will finally push all my patches. But it won't be too fast because I still have to sort out the latest ones (between 4.9.2 and 9.2.0). Both Paul's and my own.

[dmik]

All 4.9.2 patches (adapted to 9.2.0) are committed — welcome to review them. Now it's time to sort out the rest. I will also play with the build around to see how good it is.

[ydario]

Also, there is no libsupc++ as a DLL and seems to be no way to build it as such (meaning that the GCC team doesn't see any need in that). I wonder why @ydario did that manually for our GCC 4.x

I think that was built because only static libs were generated at that time using old tools. I see linux supplies supc++ only as static lib. we could do the same now, but I see a potential problem with code using supcpp6.dll; if we stop shipping it we will break such applications. I don't think they are so many, so probably rebuilding them is the best solution. #2nd solution is to merge them in stdc++ dll and provide a forwarder, but sounds hackish :-)

[ydario]

but I see a potential problem with code using supcpp6.dll;

"rpm -q --whatrequires libsupc++6" shows only gcc itself requiring it, so above text is a no-issue.

[dmik]

@ydario thank you for your comments, it makes sense now. libsupc++6 will bite the dust :) I see no need to provide a forwarder for it. But there still will be a forwarder for ssp.dll (to ssp0.dll). And most likely a forwarder for stdcpp6.dll to stdc++6.dll If I won/t find a simple way to rename it in GCC makefiles.

[dmik]

I've done all hard autoconf work by removing the pre-generated configure scripts and other stuff (the GCC project is terrible in this regard — it's all too fragile and not properly maintained). But now I'm facing an issue of CC1.EXE crashing when running some libgcc tests:

configure:3799: checking for suffix of object files
configure:3821: D:/Coding/gcc/master-build/./gcc/xgcc -BD:/Coding/gcc/master-build/./gcc/ -BD:/Coding/gcc/master-install/i386-pc-os2-emx/bin/ -BD:/Coding/gcc/master-install/i386-pc-os2-emx/lib/ -isystem D:/Coding/gcc/master-install/i386-pc-os2-emx/include -isystem D:/Coding/gcc/master-install/i386-pc-os2-emx/sys-include    -c -g -O2 -march=pentium -mtune=pentium4  conftest.c >&5

Killed by SIGSEGV
pid=0xcd0e ppid=0xcd0b tid=0x0001 slot=0x00ad pri=0x0200 mc=0x0001 ps=0x0010
D:\CODING\GCC\MASTER-BUILD\GCC\CC1.EXE
CC1 0:001aa16c
cs:eip=005b:001ba16c      ss:esp=0053:0205f870      ebp=0205f8a8
 ds=0053      es=0053      fs=150b      gs=0000     efl=00010246
eax=00000000 ebx=0205fa1c ecx=00000000 edx=00000000 edi=00000000 esi=00000000
Process dumping was disabled, use DUMPPROC / PROCDUMP to enable it.
xgcc: internal compiler error: Segmentation fault signal terminated program cc1
Please submit a full bug report,
with preprocessed source if appropriate.
See <https://gcc.gnu.org/bugs/> for instructions.
configure:3825: $? = 4

Kind of strange. The only difference with the previous build (that I did before updating from coreutils 8.26 to 8.31 and automake from 1.14 to 1.16) is that intermediate XGCC (and CC1) use the -O2 option now. (Which was not used for intermediate builds before). Needs deeper investigation.

[dmik]

What's even more strange is that it also fails in exactly the same way with the previous build of gcc where it worked before — according its config.log. The only explanation to that is that CC1.EXE was changed in some way after configuring in libgcc (which is possible since I was rebuilding stuff here and there).

Anyway, this doesn't seem to be related to the -O option. And also, the final result of the previous build (i.e. the finally cooked GCC.EXE and its CC1.EXE counter part) work. More over, if I replace this temporary CC1.EXE with its final version (which differs just by a few bytes), all works. I'm puzzled. It needs deeper debugging.

[dmik]

Fixed the above crash: it was a tiny typo in one of my (unpushed) commits to gcc/config/i386/i386.c that brings it in line with 9.2.0 affairs. So I just rebased it locally. Will give it all another ride now.

[dmik]

Finally, I built gcc with all my patches and pushed them here. This task is complete now. Any further problems are to be reported as separate tickets.

Note that in order to build it, you need a newer coreutils (because of important fixes to install) as well as a newer libc (because of the CXCHG fix, https://github.com/bitwiseworks/libc/commit/2a2eb9a47e83bc415cfa06efce1a3db2a92f9063). Note that coreutils which is in exp now has a broken ln because of another libc bug which is still to be fixed (https://github.com/bitwiseworks/libc/issues/48), but there is a test install build in the usual BWW test location (rpm.netlabs.org/test).