I got Janet working on OS X Tiger / PowerPC!

[cellularmitosis]

There were just a few missing pieces on Tiger:

• posix_spawn()
• clock_gettime()
• arc4random_buf()
• O_CLOEXEC

Work-arounds for the above are in the attached patch and activated using the following CFLAGS:

-DNO_CLOEXEC -DNO_SPAWN_H -DNO_CLOCK_GETTIME -DNO_ARC4RANDOM_BUF -DNO_POSIX_SPAWN

Details: https://gist.github.com/cellularmitosis/a54e8bf4d80fb8e3599551af6416e66a

I feel like Janet's ultra-low start-up latency and good performance make an attractive option for PowerPC Macs.

Would a PR be welcomed? Or is Tiger considered too old to officially support?

EDIT: Note: I am not 100% sure of the implications of some of these changes. For example, lacking O_CLOEXEC seems like it opens up Janet to certain race conditions. Also, I haven't at all tested the fork() / exec() replacement of posix_spawn. I'm also not sure how important CLOCK_REALTIME was.

[cellularmitosis]

tiger.patch:

diff -urN janet-1.10.1/Makefile janet-1.10.1.tiger/Makefile
--- janet-1.10.1/Makefile   2020-06-18 19:24:17.000000000 -0500
+++ janet-1.10.1.tiger/Makefile 2020-06-25 21:37:26.000000000 -0500
@@ -259,17 +259,17 @@
    cp -rf $(JANET_HEADERS) '$(DESTDIR)$(INCLUDEDIR)/janet'
    mkdir -p '$(DESTDIR)$(JANET_PATH)'
    mkdir -p '$(DESTDIR)$(LIBDIR)'
-   cp $(JANET_LIBRARY) '$(DESTDIR)$(LIBDIR)/libjanet.so.$(shell $(JANET_TARGET) -e '(print janet/version)')'
-   cp $(JANET_STATIC_LIBRARY) '$(DESTDIR)$(LIBDIR)/libjanet.a'
-   ln -sf $(SONAME) '$(DESTDIR)$(LIBDIR)/libjanet.so'
-   ln -sf libjanet.so.$(shell $(JANET_TARGET) -e '(print janet/version)') $(DESTDIR)$(LIBDIR)/$(SONAME)
+   #cp $(JANET_LIBRARY) '$(DESTDIR)$(LIBDIR)/libjanet.so.$(shell $(JANET_TARGET) -e '(print janet/version)')'
+   #cp $(JANET_STATIC_LIBRARY) '$(DESTDIR)$(LIBDIR)/libjanet.a'
+   #ln -sf $(SONAME) '$(DESTDIR)$(LIBDIR)/libjanet.so'
+   #ln -sf libjanet.so.$(shell $(JANET_TARGET) -e '(print janet/version)') $(DESTDIR)$(LIBDIR)/$(SONAME)
    cp -rf build/jpm '$(DESTDIR)$(BINDIR)'
    mkdir -p '$(DESTDIR)$(MANPATH)'
    cp janet.1 '$(DESTDIR)$(MANPATH)'
    cp jpm.1 '$(DESTDIR)$(MANPATH)'
    mkdir -p '$(DESTDIR)$(PKG_CONFIG_PATH)'
    cp build/janet.pc '$(DESTDIR)$(PKG_CONFIG_PATH)/janet.pc'
-   [ -z '$(DESTDIR)' ] && $(LDCONFIG) || true
+   #[ -z '$(DESTDIR)' ] && $(LDCONFIG) || true

 uninstall:
    -rm '$(DESTDIR)$(BINDIR)/janet'
diff -urN janet-1.10.1/src/core/net.c janet-1.10.1.tiger/src/core/net.c
--- janet-1.10.1/src/core/net.c 2020-06-18 19:24:17.000000000 -0500
+++ janet-1.10.1.tiger/src/core/net.c   2020-06-25 20:21:18.000000000 -0500
@@ -112,7 +112,7 @@
 #endif
 static JanetStream *make_stream(int fd, int flags) {
     JanetStream *stream = janet_abstract(&StreamAT, sizeof(JanetStream));
-#ifndef SOCK_CLOEXEC
+#if !defined(SOCK_CLOEXEC) && !defined(NO_CLOEXEC)
     int extra = O_CLOEXEC;
 #else
     int extra = 0;
diff -urN janet-1.10.1/src/core/os.c janet-1.10.1.tiger/src/core/os.c
--- janet-1.10.1/src/core/os.c  2020-06-18 19:24:17.000000000 -0500
+++ janet-1.10.1.tiger/src/core/os.c    2020-06-25 21:27:28.000000000 -0500
@@ -46,7 +46,9 @@
 #include <io.h>
 #include <process.h>
 #else
+#ifndef NO_SPAWN_H
 #include <spawn.h>
+#endif
 #include <utime.h>
 #include <unistd.h>
 #include <dirent.h>
@@ -68,6 +70,18 @@
  * work/link properly if we detect a BSD */
 #if defined(JANET_BSD) || defined(JANET_APPLE)
 void arc4random_buf(void *buf, size_t nbytes);
+#ifdef NO_ARC4RANDOM_BUF
+uint32_t arc4random(void);
+/* Thanks to https://stackoverflow.com/a/12956868/558735 */
+void arc4random_buf(void *buf, size_t nbytes) {
+    size_t intBufLength = (nbytes/4)+1;
+    uint32_t randomInts[intBufLength];        
+    for (size_t i = 0; i < intBufLength; i++) {
+        randomInts[i] = arc4random();
+    }
+    memcpy(buf, randomInts, nbytes);
+}
+#endif
 #endif

 /* Not POSIX, but all Unixes but Solaris have this function. */
@@ -77,6 +91,12 @@
 #define timegm _mkgmtime
 #endif

+#if defined(__MACH__) && defined(NO_CLOCK_GETTIME)
+/* _ANSI_SOURCE or _POSIX_C_SOURCE prevents these from being visible on older OS X. */
+struct tm *gmtime_r(const time_t *, struct tm *);
+struct tm *localtime_r(const time_t *, struct tm *);
+#endif
+
 /* Access to some global variables should be synchronized if not in single threaded mode, as
  * setenv/getenv are not thread safe. */
 #ifdef JANET_THREADS
@@ -388,6 +408,16 @@
     }

     pid_t pid;
+#ifdef NO_POSIX_SPAWN
+    pid = fork();
+    if (pid == 0) {
+        /* This is the child process. */
+        status = execve(child_argv[0], cargv, envp);
+        /* Note: a successful execve does not return.
+           Therefore, continued execution indicates execve has failed. */
+        exit(errno);
+    }
+#else
     if (janet_flag_at(flags, 1)) {
         status = posix_spawnp(&pid,
                               child_argv[0], NULL, NULL, cargv,
@@ -397,6 +427,7 @@
                              child_argv[0], NULL, NULL, cargv,
                              use_environ ? environ : envp);
     }
+#endif

     if (use_environ) {
         janet_unlock_environ();
@@ -522,7 +553,7 @@
     return 0;
 }
 #elif defined(__MACH__)
-static int gettime(struct timespec *spec) {
+int gettime(struct timespec *spec) {
     clock_serv_t cclock;
     mach_timespec_t mts;
     host_get_clock_service(mach_host_self(), CALENDAR_CLOCK, &cclock);
diff -urN janet-1.10.1/src/core/thread.c janet-1.10.1.tiger/src/core/thread.c
--- janet-1.10.1/src/core/thread.c  2020-06-18 19:24:17.000000000 -0500
+++ janet-1.10.1.tiger/src/core/thread.c    2020-06-25 20:56:05.000000000 -0500
@@ -234,7 +234,11 @@
     if (waiter->timedwait) {
         /* N seconds -> timespec of (now + sec) */
         struct timespec now;
+#if defined(__MACH__) && defined(NO_CLOCK_GETTIME)
+        gettime(&now);
+#else
         clock_gettime(CLOCK_REALTIME, &now);
+#endif
         time_t tvsec = (time_t) floor(sec);
         long tvnsec = (long) floor(1000000000.0 * (sec - ((double) tvsec)));
         tvsec += now.tv_sec;
diff -urN janet-1.10.1/src/core/util.h janet-1.10.1.tiger/src/core/util.h
--- janet-1.10.1/src/core/util.h    2020-06-18 19:24:17.000000000 -0500
+++ janet-1.10.1.tiger/src/core/util.h  2020-06-25 20:48:14.000000000 -0500
@@ -87,6 +87,11 @@
     int32_t argc,
     Janet *argv);

+#if defined(__MACH__)
+#include <time.h>
+int gettime(struct timespec *spec);
+#endif
+
 /* Inside the janet core, defining globals is different
  * at bootstrap time and normal runtime */
 #ifdef JANET_BOOTSTRAP

[cellularmitosis]

build-janet-tiger.sh:

#!/bin/bash

# Build and install janet-lang on OS X Tiger, as a user,
# into ~/opt/janet-x.x.x, and symlink into ~/local/bin or ~/bin.

# Note: this does not build the janet shared library (libjanet.so).
# TODO: figure out how to build a libjanet.dylib instead :)

version=1.10.1
tarball=janet-${version}.tar.gz
url=https://github.com/janet-lang/janet/archive/v${version}.tar.gz

set -e -x

mkdir -p $HOME/dist $HOME/tmp $HOME/opt/janet-$version

# Fetch:
test -e $HOME/dist/$tarball || wget -O $HOME/dist/$tarball $url

# Unpack:
cd $HOME/tmp
rm -rf janet-$version
tar xzf $HOME/dist/$tarball

# Patch:
cd janet-$version
patch -p1 < ../tiger.patch

# The default CFLAGS from janet's Makefile:
cflags="-std=c99 -Wall -Wextra -Isrc/include -Isrc/conf -fPIC -fvisibility=hidden"
# Tiger-specific build flags:
cflags="$cflags -mmacosx-version-min=10.4"
# Indicate which features Tiger is missing:
cflags="$cflags -DNO_CLOEXEC -DNO_SPAWN_H -DNO_CLOCK_GETTIME -DNO_ARC4RANDOM_BUF -DNO_POSIX_SPAWN"

# CPU-specific optimization flags.
# See /usr/include/mach/machine.h
cputype=$( sysctl hw.cputype | cut -d' ' -f2 )
cpusubtype=$( sysctl hw.cpusubtype | cut -d' ' -f2 )
if test "$cputype" = "18" -a "$cpusubtype" = "9" ; then
    cflags="$cflags -mcpu=750 -Os"   # PowerPC G3
elif test "$cputype" = "18" -a "$cpusubtype" = "10" ; then
    cflags="$cflags -mcpu=7400 -Os"  # PowerPC G4
elif test "$cputype" = "18" -a "$cpusubtype" = "11" ; then
    cflags="$cflags -mcpu=7450 -Os"  # PowerPC G4e
elif test "$cputype" = "18" -a "$cpusubtype" = "100" ; then
    cflags="$cflags -mcpu=970 -Os"   # PowerPC G5
fi

# Homebrew's MANPATH disrupts janet's Makefile.
unset MANPATH

# Note: gcc 4 and 4.2 fail with "error: thread-local storage not supported for this target".
# However, using gcc-7 (via tigerbrew's "brew install gcc") works.

# Build janet_boot faster using -O0:
make build/janet_boot \
    PREFIX=$HOME/opt/janet-$version \
    CC=gcc-7 \
    CFLAGS="$cflags -O0"

make test install \
    PREFIX=$HOME/opt/janet-$version \
    CC=gcc-7 \
    CFLAGS="$cflags"  \
    LDFLAGS=""  # Remove -rdynamic from LDFLAGS

# Symlink janet and jpm into either ~/local/bin or ~/bin.
if test -e $HOME/local/bin ; then
    ln -sf $HOME/opt/janet-$version/bin/janet $HOME/local/bin/
    ln -sf $HOME/opt/janet-$version/bin/jpm $HOME/local/bin/
elif test -e $HOME/bin ; then
    ln -sf $HOME/opt/janet-$version/bin/janet $HOME/bin/
    ln -sf $HOME/opt/janet-$version/bin/jpm $HOME/bin/
fi

[andrewchambers]

Why would macos ever be missing posix_spawn?

[cellularmitosis]

Why would macos ever be missing posix_spawn?

This page seems to have some insight into the history of posix_spawn on OS X:

_It was present in 10.5 but not many people used posixspawn, but Apple uses it a lot more internally in 10.6.

Edit: oops, forgot the link: https://lists.apple.com/archives/darwin-dev/2009/Oct/msg00081.html

[andrewchambers]

Ah thanks, I understand the history better now. So this is a discontinued type of mac, but it is still popular due to power pc support?

[cellularmitosis]

I'd like to agree, but in truth, popular is a stretch 🤣

But among the PowerPC Mac crowd, nearly all of them will be running either Tiger or Leopard. Tiger (10.4) was the last OS to support Macs with a G3, and Leopard (10.5) was the last OS to support PowerPC in general.

[bakpakin]

Thanks for the work here, this is pretty cool and I admit I haven't yet run Janet on powerpc.

Looks pretty good, some of this stuff would be good to merge, but I would like to autodetected this rather than require many build flags - a lot of this stuff seems quite manually and I will not merge it as is. I will not get in the business of maintaining patches. Basically, the four issues you identified should be options that can specified in janetconf.h (as well as meson_options.txt), perhaps with some autodetection in janet.h if they are not specified in janetconf.h, as I imagine they could apply to other platforms. Ideally, one would be able to build on tiger with only possibly editing janetconf.h, and then running make && make install. For example, to get around the thread local variable problem with the old gcc, a user could set JANET_SINGLE_THREADED in janetconf.h and build. See janetconf.h for other examples.

Next, I think building shared libraries should work in old versions of macos, not sure what the issue is here. You may .so may need to changed to .dylib everywhere, but that should be addressable. I would be willing to help clean that up and merge into the main Makefile if you can get that working.

The SOCK_CLOEXEC stuff is probably fine, as I'm not sure what we should even try to do if we can't use SOCK_CLOEXEC or O_CLOEXEC.

The clock fix looks good, I remember hitting a similar problem on older macs a long while ago (before the existence of the thread module).

I'm a little wary of your fork/exec replacement of posix spawn, so that will definitely need some more testing.

But overall, things look promising. In general, my goal here is also to add configuration option in janetconf.h to help support legacy platforms.

[cellularmitosis]

@bakpakin thanks for taking a look! I like your configuration option approach. I'll have to give some thought to autodetecting these things.

I agree, dylibs should be totally doable, I just need to learn how. I can figure that out and turn it into a PR.

[cellularmitosis]

Just wanted to follow-up with a Darwin/PowerPC status update.

On Leopard, Janet 1.27.0 builds without any patches, thanks to the MacportsLegacySupport library.

Basically:

sed -i '' -e 's|COMMON_CFLAGS:=|COMMON_CFLAGS:= -I/opt/macports-legacy-support-20221029/include/LegacySupport  |' Makefile
sed -i '' -e 's|CLIBS=|CLIBS= -L/opt/macports-legacy-support-20221029/lib  -lMacportsLegacySupport |' Makefile
make PREFIX=/opt/janet-1.27.0 'LDFLAGS=-L/opt/macports-legacy-support-20221029/lib ' CC=gcc-4.9

(details)

There is an issue when janet is built on G5 (64-bit) processors:

janet(73818) malloc: *** error for object 0x80000: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug

However, if you build janet on a G4 and then run that package on a G5, it works fine.

On Tiger, I'm still using the spawn.h / fork patch (details), which allows janet to build.

However, it fails during the test suite:

Starting suite 9...
hello
error: bad slot #0, expected string|symbol|keyword|buffer, got nil
  in string/trim [src/core/string.c] on line 577
  in _while [test/suite0009.janet] on line 33, column 28
  in _thunk [test/suite0009.janet] (tailcall) on line 28, column 1
make: *** [Makefile:228: test] Error 1

[sogaiu]

May be that refers to this line?

[cellularmitosis]

ah, thanks! looks like my half-baked spawn / fork patch needs some work :)