Build errors on Big Sur (x86_64 missing symbols, segfaults)

[rbruels]

Using the default compilers on Big Sur (macOS 11.4, Xcode 12.5.1, Intel MacBook Pro).

Installed prereqs through Homebrew.

../Retro68/build-toolchain.bash

[ 95%] Building CXX object LaunchAPPL/Client/CMakeFiles/LaunchAPPL.dir/SSH.cc.o
[ 96%] Building CXX object LaunchAPPL/Client/CMakeFiles/LaunchAPPL.dir/StreamBasedLauncher.cc.o
[ 97%] Building CXX object LaunchAPPL/Client/CMakeFiles/LaunchAPPL.dir/Serial.cc.o
[ 97%] Building CXX object LaunchAPPL/Client/CMakeFiles/LaunchAPPL.dir/TCP.cc.o
[ 98%] Building CXX object LaunchAPPL/Client/CMakeFiles/LaunchAPPL.dir/SharedFile.cc.o
[ 98%] Building CXX object LaunchAPPL/Client/CMakeFiles/LaunchAPPL.dir/Classic.cc.o
[ 99%] Building CXX object LaunchAPPL/Client/CMakeFiles/LaunchAPPL.dir/Carbon.cc.o
[ 99%] Linking CXX executable LaunchAPPL
Undefined symbols for architecture x86_64:
  "_hfs_vsetattr", referenced from:
      MiniVMacLauncher::MiniVMacLauncher(boost::program_options::variables_map&) in MiniVMac.cc.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [LaunchAPPL/Client/LaunchAPPL] Error 1
make[1]: *** [LaunchAPPL/Client/CMakeFiles/LaunchAPPL.dir/all] Error 2
make: *** [all] Error 2

Just for giggles I commented out line 234 of MiniVMac.cc:

//hfs_vsetattr(vol, &ent);

And that compile completed (obviously with unknown consequences).

Then later in the build, ConsoleTest.bin keeps segfaulting:

[  8%] Building CXX object Console/CMakeFiles/RetroConsole.dir/retro/Console.cc.obj
/Users/bruels/projects/Retro68/Console/retro/Console.cc: In member function 'bool retro::Console::ProcessEscSequence(char)':
/Users/bruels/projects/Retro68/Console/retro/Console.cc:334:34: warning: comparison of integer expressions of different signedness: 'std::basic_string<char>::size_type' {aka 'long unsigned int'} and 'const char' [-Wsign-compare]
  334 |             if(windowName.size() < MAX_LEN)    // Ignore subsequent characters
      |                ~~~~~~~~~~~~~~~~~~^~~~~~~~~
[  9%] Building CXX object Console/CMakeFiles/RetroConsole.dir/retro/ConsoleWindow.cc.obj
/Users/bruels/projects/Retro68/Console/retro/ConsoleWindow.cc: In member function 'virtual void retro::ConsoleWindow::setWindowName(std::string)':
/Users/bruels/projects/Retro68/Console/retro/ConsoleWindow.cc:78:12: warning: 'char* strncpy(char*, const char*, size_t)' specified bound 255 equals destination size [-Wstringop-truncation]
   78 |     strncpy((char *)&pname[1],newName.c_str(),255);
      |     ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[  9%] Building CXX object Console/CMakeFiles/RetroConsole.dir/retro/InitConsole.cc.obj
[ 10%] Linking CXX static library libRetroConsole.a
[ 10%] Built target RetroConsole
[ 10%] Building CXX object Console/CMakeFiles/ConsoleTest.dir/ConsoleTest.cc.obj
[ 11%] Linking CXX executable ConsoleTest.code.bin
[ 11%] Built target ConsoleTest
[ 12%] Generating ConsoleTest.bin, ConsoleTest.APPL, ConsoleTest.dsk, ConsoleTest.ad, %ConsoleTest.ad
/bin/sh: line 1: 76154 Segmentation fault: 11  /Users/bruels/projects/Retro68-build/build-host/Rez/Rez /Users/bruels/projects/Retro68/libretro/Retro68APPL.r -I/Users/bruels/projects/Retro68/libretro:/Users/bruels/projects/Retro68-build/toolchain/m68k-apple-macos/RIncludes --copy ConsoleTest.code.bin -o ConsoleTest.bin -t APPL -c ???? --cc ConsoleTest.dsk --cc ConsoleTest.APPL --cc %ConsoleTest.ad
make[2]: *** [Console/ConsoleTest.bin] Error 139
make[2]: *** Deleting file `Console/ConsoleTest.bin'
make[1]: *** [Console/CMakeFiles/ConsoleTest_APPL.dir/all] Error 2
make: *** [all] Error 2

At this point I'm stumped. Ideas appreciated!

[goshhhy]

i am also encountering both of these issues on macOS 12, on a Mac Mini M1, building for arm64.

[ryandesign]

I encounter neither of these issues on macOS 12 on Intel using prerequisites from MacPorts.

[goshhhy]

i am still encountering both on macOS 12 on Arm using prerequisites from Homebrew.

[goshhhy]

building and installing hfsutils myself and using that, rather than from homebrew, seems to fix the linker issue - so it looks like there's something up with homebrew's packaging of that, rather than Retro68.

as a workaround, just don't have hfsutils installed when you build Retro68, and it will build its own copy of hfsutils to use.

[goshhhy]

i do still get periodic segfaults from Rez, but they may be unrelated?

[rbruels]

Oh. Neat! Good find. I’ll try this in the morning and report back.

[ryandesign]

building and installing hfsutils myself and using that, rather than from homebrew, seems to fix the linker issue - so it looks like there's something up with homebrew's packaging of that, rather than Retro68.

Well you certainly can't use homebrew's hfsutils to satisfy Retro68's needs, because Retro68 needs hfsutils to have been built with the --enable-devlibs configure argument but in homebrew it isn't (while in MacPorts it is).

[timdoug]

I encountered comparable symptoms: linker not finding _hfs_vsetattr, commenting out its invocation leads to Rez segfaults. Sonoma 14.1.1 on an M2, Homebrew installed but not hfsutils.

Turns out libhfs.a from a cdrtools installation was taking precedence over the vendored copy, removing it resulted a successful build.

$ brew list | grep cdrtools
cdrtools
$ brew list | grep hfs
$ find /opt/homebrew | grep libhfs
/opt/homebrew/lib/libhfs.a
/opt/homebrew/Cellar/cdrtools/3.02a09/lib/libhfs.a
$