Implement support for Darwin ppc, fix Darwin ppc64

[barracuda156]

This PR implements initial support for Darwin PPC: adds ppc32 and fixes minor issues with ppc64 (clang should not be used for PPC on MacOS at all).

Feedback welcomed.

Some fixes to the master branch will also be needed, as it seems. I got up to here:

:info:build /opt/local/bin/gcc-mp-12 -o bin/nim c_code/1_1/stdlib_assertions.nim.o c_code/1_1/stdlib_dollars.nim.o c_code/1_1/stdlib_formatfloat.nim.o c_code/3_1/stdlib_io.nim.o c_code/3_1/stdlib_system.nim.o c_code/2_1/stdlib_parseutils.nim.o c_code/2_1/stdlib_math.nim.o c_code/1_1/stdlib_algorithm.nim.o c_code/2_1/stdlib_unicode.nim.o c_code/3_1/stdlib_strutils.nim.o c_code/2_1/stdlib_pathnorm.nim.o c_code/3_1/stdlib_posix.nim.o c_code/3_1/stdlib_times.nim.o c_code/3_1/stdlib_os.nim.o c_code/1_1/stdlib_hashes.nim.o c_code/2_1/stdlib_strtabs.nim.o c_code/1_1/stdlib_sets.nim.o c_code/2_1/@mpathutils.nim.o c_code/2_1/@mropes.nim.o c_code/3_1/stdlib_tables.nim.o c_code/2_1/@mlineinfos.nim.o c_code/3_7/@mplatform.nim.o c_code/1_1/@mprefixmatches.nim.o c_code/2_1/stdlib_strformat.nim.o c_code/2_1/stdlib_terminal.nim.o c_code/3_1/@moptions.nim.o c_code/3_1/@mmsgs.nim.o c_code/1_1/@mcondsyms.nim.o c_code/3_1/stdlib_streams.nim.o c_code/3_1/stdlib_cpuinfo.nim.o c_code/3_1/stdlib_osproc.nim.o c_code/2_3/stdlib_sha1.nim.o c_code/2_1/stdlib_lexbase.nim.o c_code/3_1/stdlib_parsejson.nim.o c_code/3_1/stdlib_json.nim.o c_code/3_1/@mextccomp.nim.o c_code/1_1/@mwordrecg.nim.o c_code/3_1/@mnimblecmd.nim.o c_code/2_3/stdlib_parseopt.nim.o c_code/1_1/@mincremental.nim.o c_code/3_1/@mcommands.nim.o c_code/2_1/@mllstream.nim.o c_code/1_1/@midents.nim.o c_code/1_1/@midgen.nim.o c_code/1_1/@mint128.nim.o c_code/1_1/@mast.nim.o c_code/1_1/@mnimlexbase.nim.o c_code/3_1/@mlexer.nim.o c_code/3_1/@mparser.nim.o c_code/1_1/@mrenderer.nim.o c_code/1_1/@mfilters.nim.o c_code/1_1/@mfilter_tmpl.nim.o c_code/1_1/@msyntaxes.nim.o c_code/1_1/stdlib_intsets.nim.o c_code/3_1/@mrodutils.nim.o c_code/3_1/@mastalgo.nim.o c_code/1_1/@mtrees.nim.o c_code/3_1/@mtypes.nim.o c_code/1_1/@mbtrees.nim.o c_code/1_1/stdlib_md5.nim.o c_code/1_1/@mmodulegraphs.nim.o c_code/1_1/@mmagicsys.nim.o c_code/1_1/@mbitsets.nim.o c_code/1_1/@mnimsets.nim.o c_code/3_1/@msemfold.nim.o c_code/3_1/@mmodulepaths.nim.o c_code/1_1/@mvmdef.nim.o c_code/3_1/@msemdata.nim.o c_code/1_1/@mlinter.nim.o c_code/1_1/@mnimfix@sprettybase.nim.o c_code/1_1/@mlookups.nim.o c_code/3_1/@msemtypinst.nim.o c_code/1_1/@mparampatterns.nim.o c_code/1_1/@mlowerings.nim.o c_code/1_1/@m..@slib@spackages@sdocutils@srstast.nim.o c_code/3_1/@m..@slib@spackages@sdocutils@srst.nim.o c_code/1_1/@m..@slib@spackages@sdocutils@shighlite.nim.o c_code/3_1/@m..@slib@spackages@sdocutils@srstgen.nim.o c_code/1_1/stdlib_xmltree.nim.o c_code/1_1/stdlib_uri.nim.o c_code/1_1/stdlib_cgi.nim.o c_code/1_1/@mtypesrenderer.nim.o c_code/3_1/@mdocgen.nim.o c_code/3_1/@msigmatch.nim.o c_code/1_1/@mimporter.nim.o c_code/1_1/@mprocfind.nim.o c_code/3_1/@mpragmas.nim.o c_code/3_1/@mreorder.nim.o c_code/1_1/@mpasses.nim.o c_code/1_1/@msaturate.nim.o c_code/1_1/@mguards.nim.o c_code/3_1/@msighashes.nim.o c_code/3_1/@mliftdestructors.nim.o c_code/1_1/@msempass2.nim.o c_code/1_1/@mcgmeth.nim.o c_code/1_1/@maliases.nim.o c_code/1_1/@mpatterns.nim.o c_code/3_1/@mdfa.nim.o c_code/3_1/@minjectdestructors.nim.o c_code/1_1/@mliftlocals.nim.o c_code/1_1/@mlambdalifting.nim.o c_code/1_1/@mclosureiters.nim.o c_code/1_1/@mtransf.nim.o c_code/1_1/@mvmgen.nim.o c_code/3_1/@mvmdeps.nim.o c_code/1_1/@mvmmarshal.nim.o c_code/3_1/@mgorgeimpl.nim.o c_code/1_1/@mmacrocacheimpl.nim.o c_code/3_1/@mevaltempl.nim.o c_code/3_1/@mvm.nim.o c_code/1_1/@msemmacrosanity.nim.o c_code/1_1/@mpluginsupport.nim.o c_code/1_1/@mplugins@slocals.nim.o c_code/1_1/@mplugins@sitersgen.nim.o c_code/1_1/@mplugins@sactive.nim.o c_code/1_1/@mspawn.nim.o c_code/1_1/@msemparallel.nim.o c_code/3_1/@msem.nim.o c_code/1_1/@mccgutils.nim.o c_code/1_1/@mtreetab.nim.o c_code/1_1/@mndi.nim.o c_code/1_1/@mcgendata.nim.o c_code/3_1/@mccgmerge.nim.o c_code/1_1/@menumtostr.nim.o c_code/2_1/stdlib_dynlib.nim.o c_code/3_1/@mcgen.nim.o c_code/3_1/@mnimconf.nim.o c_code/1_1/@mpassaux.nim.o c_code/1_1/@mdepends.nim.o c_code/1_1/@mmodules.nim.o c_code/1_1/@mjsgen.nim.o c_code/3_1/@mdocgen2.nim.o c_code/1_1/@mmain.nim.o c_code/3_1/@mscriptconfig.nim.o c_code/3_1/@mcmdlinehelper.nim.o c_code/3_1/@mnim.nim.o -ldl -lm -arch ppc
:info:build : SUCCESS

But then the main code again invokes Clang, which fails, of course. I will fix that in another PR to Nim master branch.

[barracuda156]

P. S. For Rosetta, this part poses a problem and breaks the build:

    if [ "$HOSTTYPE" = "x86_64" ] ; then
      ucpu="amd64"
    fi

I do not know how to fix it in a generally acceptable way at the moment. Locally I just did this:

    if [ "$HOSTTYPE" = "x86_64" ] ; then
      ucpu="ppc"
    fi

This works, but obviously cannot be merged, since it gonna break all Intel builds. If anyone can suggest how to write a special case for Rosetta, it would be greatly appreciated. (On Rosetta the physical cpu is x86*, but the triple is powerpc-apple-darwin10, since ppc32 is emulated by the OS itself.)

[barracuda156]

@Araq When you have time, please take a look. I would like to bring Nim to MacOS PPC, and that should be feasible, provided ppc64 exists (at least potentially – it certainly could not have been built on MacOS with Clang).

(Rosetta issue I mention in the comment above may be safely ignored: I can add a patch for it in Macports when I update Nim port.)

[Araq]

Sorry for this rather late reply, but how was this done? I need to replicate your steps.

[barracuda156]

@Araq Thanks for replying, and I will greatly appreciate your involvement here.

To the question: I started from assumption that existing code base is coherent and tested (ppc64 on Mac might not have been, but that is not essential in this context), extracted lists of sources used from build.sh for combinations of platforms and archs, diffed them, turned out there are not many files which are changed within a platform depending on an arch, and most of those are shared within an arch across platforms (example stdlib_sha1.nim.c). The only one unique was @mplatform.nim.c, but diffing that, only code number of processor is changed for a given OS (compare amd64 and ppc64 for Mac), so that file I had to make, and put it into 3_7 (existing folders already have some version of that file).

--- /Users/svacchanda/Desktop/ppc64_mac.sh  2022-09-15 21:42:25.000000000 +0700
+++ /Users/svacchanda/Desktop/amd64_mac.sh  2022-09-15 21:51:54.000000000 +0700
@@ -1,10 +1,10 @@
-  powerpc64)
+  amd64)
     set -x
     $CC $COMP_FLAGS -Ic_code -c c_code/1_2/stdlib_assertions.nim.c -o c_code/1_2/stdlib_assertions.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/1_2/stdlib_dollars.nim.c -o c_code/1_2/stdlib_dollars.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/1_2/stdlib_formatfloat.nim.c -o c_code/1_2/stdlib_formatfloat.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/3_2/stdlib_io.nim.c -o c_code/3_2/stdlib_io.nim.o
-    $CC $COMP_FLAGS -Ic_code -c c_code/3_6/stdlib_system.nim.c -o c_code/3_6/stdlib_system.nim.o
+    $CC $COMP_FLAGS -Ic_code -c c_code/3_2/stdlib_system.nim.c -o c_code/3_2/stdlib_system.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/2_2/stdlib_parseutils.nim.c -o c_code/2_2/stdlib_parseutils.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/2_2/stdlib_math.nim.c -o c_code/2_2/stdlib_math.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/1_2/stdlib_algorithm.nim.c -o c_code/1_2/stdlib_algorithm.nim.o
@@ -21,7 +21,7 @@
     $CC $COMP_FLAGS -Ic_code -c c_code/2_2/@mropes.nim.c -o c_code/2_2/@mropes.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/3_2/stdlib_tables.nim.c -o c_code/3_2/stdlib_tables.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/2_4/@mlineinfos.nim.c -o c_code/2_4/@mlineinfos.nim.o
-    $CC $COMP_FLAGS -Ic_code -c c_code/3_6/@mplatform.nim.c -o c_code/3_6/@mplatform.nim.o
+    $CC $COMP_FLAGS -Ic_code -c c_code/3_2/@mplatform.nim.c -o c_code/3_2/@mplatform.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/1_2/@mprefixmatches.nim.c -o c_code/1_2/@mprefixmatches.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/2_4/stdlib_strformat.nim.c -o c_code/2_4/stdlib_strformat.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/2_2/stdlib_terminal.nim.c -o c_code/2_2/stdlib_terminal.nim.o
@@ -31,14 +31,14 @@
     $CC $COMP_FLAGS -Ic_code -c c_code/3_2/stdlib_streams.nim.c -o c_code/3_2/stdlib_streams.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/3_2/stdlib_cpuinfo.nim.c -o c_code/3_2/stdlib_cpuinfo.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/3_2/stdlib_osproc.nim.c -o c_code/3_2/stdlib_osproc.nim.o
-    $CC $COMP_FLAGS -Ic_code -c c_code/2_6/stdlib_sha1.nim.c -o c_code/2_6/stdlib_sha1.nim.o
+    $CC $COMP_FLAGS -Ic_code -c c_code/3_2/stdlib_sha1.nim.c -o c_code/3_2/stdlib_sha1.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/2_2/stdlib_lexbase.nim.c -o c_code/2_2/stdlib_lexbase.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/3_2/stdlib_parsejson.nim.c -o c_code/3_2/stdlib_parsejson.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/3_2/stdlib_json.nim.c -o c_code/3_2/stdlib_json.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/3_2/@mextccomp.nim.c -o c_code/3_2/@mextccomp.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/1_2/@mwordrecg.nim.c -o c_code/1_2/@mwordrecg.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/3_2/@mnimblecmd.nim.c -o c_code/3_2/@mnimblecmd.nim.o
-    $CC $COMP_FLAGS -Ic_code -c c_code/2_4/stdlib_parseopt.nim.c -o c_code/2_4/stdlib_parseopt.nim.o
+    $CC $COMP_FLAGS -Ic_code -c c_code/2_2/stdlib_parseopt.nim.c -o c_code/2_2/stdlib_parseopt.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/1_2/@mincremental.nim.c -o c_code/1_2/@mincremental.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/3_2/@mcommands.nim.c -o c_code/3_2/@mcommands.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/2_2/@mllstream.nim.c -o c_code/2_2/@mllstream.nim.o

Of these, stdlib_system.nim.c differ for 64-bit versions but not for 32-bit versions of OS (for a given platform). So I used i386 for Mac as a template and replaced arch-specific files with those for ppc.

(Apparently organization of files into folders was changed between older csources packed within Nim 1.6.6 distro and these ones and few extra files been added, so my patch will not work with the former. I am using the latest-commit version of csources_v1 with my patches on top.)

Then, Clang should not be used for any PPC on Mac, at least until someone has time and motivation to fix it eventually. So Clang, if hardcoded for w/e reason, should not be used for PPC. GCC works. st_mtim patch borrowed from here: https://github.com/marlam/mpop-mirror/issues/9#issuecomment-940777826

At this point, bootstrapping from csources_v1 appears to work fine, while subsequent compilation of Nim source fails on static asserts here: https://github.com/nim-lang/Nim/issues/19914#issuecomment-1251268726 (notice the same issue occurred on other Unix). That could be some bug in the code (I assume a less likely alternative), Rosetta-specific problem (Rosetta gets tricky to fix sometimes), or I still need to fix something in the source that was missed (this is not relevant to this PR).

[barracuda156]

@Araq There are few other changes needed to replicate the process on 10.6.8 Rosetta and use a newer GCC which are installed in Macports prefix. That we can handle locally in Macports, since I do not know how to fix it in a generally correct way.

This should only be used on Rosetta (it is a hack, since build system [mis]detects host arch as x86):

--- a/build.sh  2022-05-05 07:55:19.000000000 +0700
+++ b/build.sh  2022-09-16 00:34:38.000000000 +0700
@@ -127,7 +127,7 @@
     CC="clang"
     LINK_FLAGS="$LINK_FLAGS -ldl -lm"
     if [ "$HOSTTYPE" = "x86_64" ] ; then
-      ucpu="amd64"
+      ucpu="ppc"
     fi
     ;;
   *aix* )

--- a/makefile  2022-05-05 07:55:19.000000000 +0700
+++ b/makefile  2022-09-16 23:26:53.000000000 +0700
@@ -47,7 +47,7 @@
   CC = clang
   LDFLAGS += -ldl -lm
   ifeq ($(HOSTTYPE),x86_64)
-    ucpu = amd64
+    ucpu = ppc
   endif
 endif
 ifeq ($(uos),aix)

--- a/tools/niminst/buildsh.nimf    2022-05-04 23:15:31.000000000 +0700
+++ b/tools/niminst/buildsh.nimf    2022-09-17 20:29:34.000000000 +0700
@@ -128,7 +128,7 @@
     CC="clang"
     LINK_FLAGS="$LINK_FLAGS -ldl -lm"
     if [ "$HOSTTYPE" = "x86_64" ] ; then
-      ucpu="amd64"
+      ucpu="ppc"
     fi
     ;;
   *aix* )

--- a/tools/niminst/makefile.nimf   2022-05-04 23:15:31.000000000 +0700
+++ b/tools/niminst/makefile.nimf   2022-09-17 20:30:49.000000000 +0700
@@ -50,7 +50,7 @@
   CC = clang
   LDFLAGS += -ldl -lm
   ifeq ($(HOSTTYPE),x86_64)
-    ucpu = amd64
+    ucpu = ppc
   endif
 endif
 ifeq ($(uos),aix)

And this must be patched to avoid invoking gcc-4.2 which does not accept fmax-errors=:

--- a/config/nim.cfg    2022-05-04 23:15:31.000000000 +0700
+++ b/config/nim.cfg    2022-09-19 23:03:47.000000000 +0700
@@ -9,6 +9,8 @@
 #  gcc.path %= "$CC_PATH"

 cc = gcc
+gcc.exe %= "@gcc_v@"
+gcc.path %= "@prefix@"

 # additional options always passed to the compiler:
 --parallel_build: "0" # 0 to auto-detect number of processors

In all patches from both PRs gcc changed to @GCC@, then reinplace used in portfile to set correct paths to /opt/local/bin/gcc-mp-12 (or whichever chosen version). Something like:

    post-patch {
        reinplace "s|@GCC@|${configure.cc}|g" ${worksrcpath}/build.sh
        reinplace "s|@GCC@|${configure.cc}|g" ${worksrcpath}/makefile
        reinplace "s|@GCC@|${configure.cc}|g" ${worksrcpath}/tools/niminst/buildsh.nimf
        reinplace "s|@GCC@|${configure.cc}|g" ${worksrcpath}/tools/niminst/makefile.nimf

        if {[regexp {macports-gcc-(.*)} ${configure.compiler} -> gcc_v]} {
            reinplace "s|@gcc_v@|gcc-mp-${gcc_v}|g" ${worksrcpath}/config/nim.cfg
            reinplace "s|@prefix@|${prefix}/bin|g" ${worksrcpath}/config/nim.cfg
        } else {
            reinplace "s|@gcc_v@|gcc|g" ${worksrcpath}/config/nim.cfg
            reinplace "s|@prefix@|usr/bin|g" ${worksrcpath}/config/nim.cfg
        }
    }

[Araq]

Uh, but you're supposed to use the "niminst" tool and adapt the configuration in compiler/installer.ini :-)

[Varriount]

@Araq Is that documented anywhere? And if it isn't, @barracuda156, could you add documentation (wherever is sensible) to modify niminst instead?

[barracuda156]

Uh, but you're supposed to use the "niminst" tool and adapt the configuration in compiler/installer.ini :-)

@Araq Thank you, I will check that.

(Please allow me a few days, I am in a mid of a travel until the end of month – and I also get back to my PPC machines by then.)

[barracuda156]

Uh, but you're supposed to use the "niminst" tool and adapt the configuration in compiler/installer.ini :-)

@Araq Do I get it right that to use niminst one has to have working Nim to begin with? :)

I do not have it yet, I need to build it first (and there is an assert error on PPC after csources are built and koch initialized).

[Araq]

You only need a working Nim on some machine, it doesn't have to be the PPC thing.

[barracuda156]

You only need a working Nim on some machine, it doesn't have to be the PPC thing.

@Araq Ok, I will try it.