Error Building mRuby on MacOS

[gyrovorbis]

We've already discussed this in Discord, but I wanted to make an issue to track it, so that random people who encounter the issue on MacOS can see we're on it.

But... yeah, mRuby from kos-ports builds fine for Linux and Windows, but is failing for me with the following error:

YACC  mrbgems/mruby-compiler/core/parse.y -> mrbgems/mruby-compiler/core/y.tab.c
/opt/toolchains/dc/kos-ports/mruby/build/mruby-3.2.0/mrbgems/mruby-compiler/core/parse.y:1449.9-19: syntax error, unexpected identifier, expecting string
rake aborted!
Command failed with status (1): [bison -o "/opt/toolchains/dc/kos-ports/mru...]
/opt/toolchains/dc/kos-ports/mruby/build/mruby-3.2.0/lib/mruby/build/command.rb:33:in `_run'
/opt/toolchains/dc/kos-ports/mruby/build/mruby-3.2.0/lib/mruby/build/command.rb:259:in `run'
/opt/toolchains/dc/kos-ports/mruby/build/mruby-3.2.0/mrbgems/mruby-compiler/mrbgem.rake:23:in `block in <top (required)>'
/Library/Ruby/Gems/2.6.0/gems/rake-12.3.3/exe/rake:27:in `<top (required)>'
Tasks: TOP => default => all => gensym => /opt/toolchains/dc/kos-ports/mruby/build/mruby-3.2.0/build/dreamcast/presym => /opt/toolchains/dc/kos-ports/mruby/build/mruby-3.2.0/build/dreamcast/mrbgems/mruby-compiler/core/y.tab.pi => /opt/toolchains/dc/kos-ports/mruby/build/mruby-3.2.0/mrbgems/mruby-compiler/core/y.tab.c
(See full trace by running task with --trace)
make[2]: *** [all] Error 1
make[1]: *** [generatemruby] Error 2
make: *** [build-stamp] Error 2
2
Error building /opt/toolchains/dc/kos/../kos-ports/mruby. Bailing out.
falco@hackintosh utils % 

This is a Hackintosh VM on Intel silicon, but fairly certain @darcagn saw the exact same issue on Apple silicon.

Seems to me that mRuby itself doesn't seem like it'll build on MacOS right now for anyone with this, and that it's not Dreamcast-specific? Not sure. Tagging @sizious to keep him in the loop.

[sizious]

Thank you for reporting the issue @gyrovorbis! Indeed, I asked matz here: https://github.com/mruby/mruby/issues/6135#issuecomment-1872622268

Unfortunately, I don't own a macOS machine as well so I'm a bit stuck here.

[gyrovorbis]

Thank you for reporting the issue @gyrovorbis! Indeed, I asked matz here: mruby/mruby#6135 (comment)

Unfortunately, I don't own a macOS machine as well so I'm a bit stuck here.

Oh, thanks so much for opening that conversation! Let me give this a try with a newer version of Bison on my VM!

[gyrovorbis]

Okay! So brew install bison to install a newer version of Bison did indeed fix the error we were encountering -- provided you also added Brew's Bison to your ENV path before the builtin XCode Bison or whatever ships with MacOS...

However, it seems that there's still a final issue with the install after everything else builds fine?

CC    build/dreamcast/mrbgems/mruby-proc-binding/gem_init.c -> build/dreamcast/mrbgems/mruby-proc-binding/gem_init.o
CC    mrbgems/mruby-sleep/src/sleep.c -> build/dreamcast/mrbgems/mruby-sleep/src/sleep.o
CC    build/dreamcast/mrbgems/mruby-sleep/gem_init.c -> build/dreamcast/mrbgems/mruby-sleep/gem_init.o
CC    build/dreamcast/mrbgems/gem_init.c -> build/dreamcast/mrbgems/gem_init.o
AR    build/dreamcast/lib/libmruby.a 
GEN   build/host/bin/mrbc -> bin/mrbc

Build summary:

================================================
      Config Name: dreamcast
 Output Directory: build/dreamcast
    Included Gems:
             mruby-array-ext - Array class extension
             mruby-bigint - Integer class extension to multiple-precision
             mruby-binding - Binding class (core features only)
             mruby-catch - Catch / Throw non-local Jump
             mruby-class-ext - class/module extension
             mruby-cmath - standard Math module with complex
             mruby-compar-ext - Comparable module extension
             mruby-compiler - mruby compiler library
             mruby-complex - Complex class
             mruby-data - standard Data class
             mruby-enum-chain - Enumerator::Chain class
             mruby-enum-ext - Enumerable module extension
             mruby-enum-lazy - Enumerator::Lazy class
             mruby-enumerator - Enumerator class
             mruby-errno - Errno module and SystemCallError class
             mruby-error - extensional error handling
             mruby-eval - standard Kernel#eval method
             mruby-exit - Kernel#exit method
             mruby-fiber - Fiber class
             mruby-hash-ext - Hash class extension
             mruby-kernel-ext - extensional function-like methods
             mruby-math - standard Math module
             mruby-metaprog - Meta-programming features for mruby
             mruby-method - Method and UnboundMethod class
             mruby-numeric-ext - Numeric class extension
             mruby-object-ext - extensional methods shared by all objects
             mruby-objectspace - ObjectSpace class
             mruby-os-memsize - ObjectSpace memsize_of method
             mruby-pack - Array#pack and String#unpack method
             mruby-print - standard print/puts/p
             mruby-proc-binding - Proc#binding method
             mruby-proc-ext - Proc class extension
             mruby-random - Random class
             mruby-range-ext - Range class extension
             mruby-rational - Rational class
             mruby-set
             mruby-sleep - Kernel#sleep and Kernel#usleep
             mruby-sprintf - standard Kernel#sprintf method
             mruby-string-ext - String class extension
             mruby-struct - standard Struct class
             mruby-symbol-ext - Symbol class extension
             mruby-time - standard Time class
             mruby-toplevel-ext - toplevel object (main) methods extension
================================================

================================================
      Config Name: host
 Output Directory: build/host
         Binaries: mrbc
    Included Gems:
             mruby-bin-mrbc - mruby compiler executable
             mruby-compiler - mruby compiler library
================================================

Updating includes before installation...
Installing mrbc to /opt/toolchains/dc/sh-elf/../bin...
error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/strip: can't open file: /opt/toolchains/dc/sh-elf/../bin/mrbc (Not a directory)
make[1]: *** [installmrbc] Error 1
make: *** [build-stamp] Error 2
2
Error building /opt/toolchains/dc/kos/../kos-ports/mruby. Bailing out.

Notice everything builds fine and mrbc gets put into here: GEN build/host/bin/mrbc -> bin/mrbc while it seems like the next command expects it here:

Installing mrbc to /opt/toolchains/dc/sh-elf/../bin...
error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/strip: can't open file: /opt/toolchains/dc/sh-elf/../bin/mrbc (Not a directory)

[gyrovorbis]

Tagging @darcagn, because I'm assuming part of this is going to be adding bison to the list of prereqs he has people install on MacOS with his tutorial then adding it to the ENV path before the system/XCode paths.

[sizious]

That’s my fault, I forgot to add a mkdir instruction before copying mrbc… indeed in my computer this directory, /opt/toolchains/dc/bin was created by dc-tool, while installing it. So you may just add the mkdir instruction before the copy and it should fix the issue.

[sizious]

Here: https://github.com/KallistiOS/kos-ports/blob/499fe42f62ae8b82768cb31684d96d409c1cb300/mruby/files/KOSMakefile.mk#L44 It just need the following: mkdir -p $(MRUBY_INSTALL_DIR)

[gyrovorbis]

Here:

https://github.com/KallistiOS/kos-ports/blob/499fe42f62ae8b82768cb31684d96d409c1cb300/mruby/files/KOSMakefile.mk#L44

It just need the following: mkdir -p $(MRUBY_INSTALL_DIR)

Makes perfect sense to me, but I made the change and am getting the following:

================================================
      Config Name: host
 Output Directory: build/host
         Binaries: mrbc
    Included Gems:
             mruby-bin-mrbc - mruby compiler executable
             mruby-compiler - mruby compiler library
================================================

Updating includes before installation...
Installing mrbc to /opt/toolchains/dc/sh-elf/../bin...
mkdir -p /opt/toolchains/dc/sh-elf/../bin
mkdir: /opt/toolchains/dc/sh-elf/../bin: File exists
make[1]: *** [installmrbc] Error 1
make: *** [build-stamp] Error 2
2
Error building /opt/toolchains/dc/kos/../kos-ports/mruby. Bailing out.
falco@hackintosh utils % 

But I don't understand it... the -p flag should make it not care if a directory is present? It definitely does when issued from the command-line? SURELY I'm doing something stupid here?

TARGET = libmruby.a

# Definition of build directories
MRUBY_BUILD_DIR = $(CURDIR)/build
MRUBY_BUILD_KOS_ARCH_DIR = $(MRUBY_BUILD_DIR)/$(KOS_ARCH)

# Definition of headers directory (as we will need to alter the relations)
MRUBY_INCLUDE_DIR = $(MRUBY_BUILD_KOS_ARCH_DIR)/include
MRUBY_INCLUDE_MRUBY_DIR = $(MRUBY_INCLUDE_DIR)/mruby
MRUBY_INCLUDE_PRESYM_DIR = $(MRUBY_INCLUDE_MRUBY_DIR)/presym

# Handling mruby compiler executable (mrbc)
# This compiler is necessary to transpile Ruby sources into a C source file
MRUBY_COMPILER = mrbc
MRUBY_INSTALL_DIR = $(KOS_CC_BASE)/../bin

# Windows specific
ifeq ($(OS),Windows_NT)
        # On Windows, all executables got a file extension
        EXECUTABLEEXTENSION = .exe

        # "sed" can display some useless warning related to "permission denied".
        # We will just hide this, as this is not relevant.
        SED_FLAGS = >/dev/null 2>&1
endif

# mruby compiler executable (mrbc) (with extension if necessary)
MRUBY_COMPILER_BIN = $(MRUBY_COMPILER)$(EXECUTABLEEXTENSION)

defaultall: | generatemruby fixincludes copylib installmrbc

# Generate mruby
generatemruby:
        @echo "Generating mruby..."
        $(MAKE) MRUBY_CONFIG=dreamcast_shelf

# Copy final library file into the root directory
copylib:
        @cp $(MRUBY_BUILD_KOS_ARCH_DIR)/lib/$(TARGET) $(CURDIR)

# Install mruby compiler executable (mrbc)
installmrbc:
        @echo "Installing $(MRUBY_COMPILER) to $(MRUBY_INSTALL_DIR)..."
        mkdir -p $(MRUBY_INSTALL_DIR)
        @cp $(MRUBY_BUILD_DIR)/host/bin/$(MRUBY_COMPILER_BIN) $(MRUBY_INSTALL_DIR)
        @strip $(MRUBY_INSTALL_DIR)/$(MRUBY_COMPILER_BIN)

# Alter header files and alter #include directives
#       #include "{variable}" => #include <mruby/{variable}>
#       #include "<mruby/{variable}>" => #include <mruby/mruby/{variable}>
#       #include "mrbconf.h" => #include <mruby/mrbconf.h> (in "mruby.h")
fixincludes:
        @echo "Updating includes before installation..."
        @for _file in $(MRUBY_INCLUDE_DIR)/*.h $(MRUBY_INCLUDE_MRUBY_DIR)/*.h $(MRUBY_INCLUDE_PRESYM_DIR)/*.h; do \
                sed -ri -e 's/#include "([^[:space:]]+)"/#include <mruby\/\1>/g;s/#include <(mruby[^[:space:]]+)>/#include <mruby\/\1>/g' $$_file $(SED_FLAGS); \
        done
        @for _file in $(MRUBY_INCLUDE_DIR)/*.h; do \
                sed -i -e 's/#include <mruby\/mruby\/mrbconf.h>/#include <mruby\/mrbconf.h>/g' $$_file $(SED_FLAGS); \
        done

[sizious]

Thanks for the feedback! I finally did the necessary to have a newer macOS VM. First of all I confirm that the update of Bison indeed solve the issue, as explained here.

For the issue you have, I think it's linked to the previous try you made to install the port. Indeed I guess you have a file at this location, i.e., /opt/toolchains/dc/bin instead of a directory. Can you check that?

On my end, I first installed dc-load so the directory /opt/toolchains/dc/bin has been created. While trying to install the mruby port, it works:

[sizious]

FYI mruby 3.3.0 is currently in rc2, so I guess we need to wait to put the 3.3.0 tag in the mruby Makefile.

[darcagn]

I'm seeing the bin error on FreeBSD:

Updating includes before installation...
Installing mrbc to /opt/toolchains/dc/sh-elf/../bin...
strip: open /opt/toolchains/dc/sh-elf/../bin/mrbc failed: Not a directory
gmake[1]: *** [KOSMakefile.mk:45: installmrbc] Error 1

/opt/toolchains/dc/bin is indeed a file, not a directory. If I rm bin, a new bin file is created somehow during building mRuby, and it still fails. If I rm bin and mkdir bin before running the mRuby install, it installs successfully.

[sizious]

Ok thanks, I'll make the necessary and make the required PR. 😊

[sizious]

OK, here we go: https://github.com/KallistiOS/kos-ports/pull/47

[gyrovorbis]

@darcagn Can you confirm that it now works on the master branch? I tested on MacOS and Ubuntu,and everything seems fine. I even had the same error as you on MacOS, and now it appears to be fixed.

[darcagn]

It works on FreeBSD now, too. Closing issue as solved.