v3.04a universal binaries not working under silicon Macos

[darth-hp]

Hi,

I downloaded wit-v3.04a-r8427-mac.tar.gz and run the installer. I do see all files are universal binaries but unfortunately the system complaints that the files are broken and can't be run.

I then downloaded wit-v3.03a-r8245-mac.tar.gz as x86_64 and this works.

file shows arm64 as architecture for the v3.04a binaries where other universal binaries I have show arm64e.

[Wiimm]

This will change with the next release.

[impierced]

Looks like wit-v3.05a still doesn't work for me on Apple Silicon. The issue appears to be with the arm64 binary as the x86_64 binary works fine.

me@system bin % ./wit zsh: killed ./wit

me@system bin % lipo -archs wit x86_64 arm64 me@system bin % lipo -remove arm64 wit -output wit_x86_64 me@system bin % ./wit_x86_64

    *****************************************
    *    __            __ _ ___________     *
    *    \ \          / /| |____   ____|    *
    *     \ \        / / | |    | |         *
    *      \ \  /\  / /  | |    | |         *
    *       \ \/  \/ /   | |    | |         *
    *        \  /\  /    | |    | |         *
    *         \/  \/     |_|    |_|         *
    *                                       *
    *           Wiimms ISO Tools            *
    *         https://wit.wiimm.de/         *
    *                                       *
    *****************************************

wit: Wiimms ISO Tool v3.05a r8638 mac/x64 - Dirk Clemens - 2022-08-27 Visit https://wit.wiimm.de/wit for more info.

-> Type 'wit_x86 -h' or 'wit_x86 help' (pipe it to a pager like 'less') for more help.

[impierced]

I simply modified the Makefile (quick and very dirty) from the source:

ifeq ($(SYSTEM),mac) CFLAGS += -I/usr/local/include CFLAGS += -arch arm64 -arch x86_64 else LDFLAGS += -static-libgcc endif

Unsure why, but I had to "export SYSTEM=mac", but I was able to then use "make" to create a universal binary of v3.05a that worked.

[Wiimm]

My problem is, that I haven't any access to a Mac and github doesn't support arm64. But I need github to test a compiler run for Mac. A friend of mine creates the universal binary at home. Because of this situation, the makefile doesn't support arm64.

Your told "not working": Do you mean the binaries created by my friend, or self compiled binaries?

[impierced]

The binaries created by your friend are not working. When I modified the Makefile and compiled it myself, those work just fine.

Happy to upload/send you the universal binaries that I compile.

[Wiimm]

Can you please do a test for me?

1. Enter the wit project directory (place, where system.c exists: ./project) and execute gcc -E -DPRINT_SYSTEM_SETTINGS system.c. It won't compile anything, but do some preprocessor stuff.
2. Load http://download.wiimm.de/misc/2022-10/mac-test.zip and unzip. Enter directory mac-test and execute same command.
3. Finally please post both outputs.

[darth-hp]

➜  project git:(master) gcc -E -DPRINT_SYSTEM_SETTINGS system.c
# 1 "system.c"
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 400 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "system.c" 2
# 37 "system.c"
# 1 "./dclib/dclib-system.h" 1
# 136 "./dclib/dclib-system.h"
result_SYSTEM="mac"
result_SYSTEM2="mac/arm"
result_SYSTEM_LINUX=0
# 38 "system.c" 2

➜  mac-test git:(master) ✗ gcc -E -DPRINT_SYSTEM_SETTINGS system.c
# 1 "system.c"
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 400 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "system.c" 2
# 37 "system.c"
# 1 "./dclib/dclib-system.h" 1
# 141 "./dclib/dclib-system.h"
result_SYSTEM="mac"
result_SYSTEM2="mac/arm"
result_SYSTEM_LINUX=0
# 38 "system.c" 2

[Wiimm]

Then I don't understand why you have to export SYSTEM=mac. The old self-check already defines it by result_SYSTEM="mac" (result_ is removed by script). So my update won't help. :(

Anyway, thanks for testing.

[Leseratte10]

I have compiled most of the previous WIT and WSZST tools Mac releases using a nonstandard Mac cross-compiler on Linux, because neither Wiimm nor me have proper access to a Mac machine.

The only difference between arm64 and arm64e is that the latter uses / requires pointer authentication, so that's most likely not going to be the issue.

Is there nothing more useful in the system logs or some other apple error logs? Just "Killed" in the terminal? Maybe this is the typical Mac error for unsigned binaries? Does this help?

If the Intel version is working just fine, maybe we should just drop the ARM version then and rely on Rosetta 2 on the ARM macs to run Wiimms tools; unless someone with more Mac knowledge figures out what exactly is wrong with the binaries.

[impierced]

The issue seems to be that the SYSTEM variable isn't getting written to the Makefile.setup file.

The awk function gensub doesn't appear to be available in macOS, "awk: calling undefined function gensub".

From setup.sh: gcc $xflags -E -DPRINT_SYSTEMSETTINGS system.c \ | awk -F= '/^result/ {printf("%s := %s\n",substr($1,8),gensub(/"/,"","g",$2))}' \

Makefile.setup

[impierced]

Leseratte10, I can compile an ARM version (or universal binary) and it works just fine.

Edit: I see you posted more information about the compiler being used. I'm compiling natively on macOS with Xcode. This explains why the function gensub is working for you and not for me.

M1 binaries shouldn't require a recompile for M2 chips, I was throwing whatever I could think of out there.

[Leseratte10]

The compiler we're using is apple-darwin20.1-clang with MACOSX_DEPLOYMENT_TARGET=10.10 and the SDK version 11.0; using osxcross from June.

Do we need a newer SDK for M2 Macs? I don't know that much about MacOS but I assumed that M2 devices should be able to run binaries from an M1 Mac?

[impierced]

I just downloaded the 3.05a universal binary. I'm getting a different error than I did with the 3.04a universal binary (that appears to have been removed).

% ./wit macOS pops up a dialog box with: "'wit' is damaged and can’t be opened. You should move it to the Trash."

Using "spctl --add wit" didn't have any effect on the binary.

% lipo -archs wit x86_64 arm64

Extracting out the x86_64 binary worked, so it's still an issue with the arm64 binary.

[Leseratte10]

I don't know much about MacOS so maybe that's what you already tried, but what happens when you open the folder in Finder, then Ctrl+click on the binary and then select "Open" from the context menu? Does that give you an option to override that, and if so, does that open the ARM binary?

We have one compiler call with CFLAGS="-arch x86_64 -arch arm64 so it should be using the same settings for both architectures ...

[Wiimm]

The SYSTEM=mac bug is solved in my working copy. I replaced line 73 of file setup.sh from | awk -F= '/^result_/ {printf("%s := %s\n",substr($1,8),gensub(/"/,"","g",$2))}' \ to | awk -F= '/^result_/ { gsub(/"/,"",$2); printf("%s := %s\n",substr($1,8),$2) }' \

[impierced]

"...what happens when you open the folder in Finder, then Ctrl+click on the binary and then select "Open" from the context menu? Does that give you an option to override that, and if so, does that open the ARM binary?"

I'm prompted to allow the app to run ... I authenticate as an admin ... and it fails like the 3.04a universal binary did.

[impierced]

The SYSTEM=mac bug is solved in my working copy. I replaced line 73 of file setup.sh from | awk -F= '/^result_/ {printf("%s := %s\n",substr($1,8),gensub(/"/,"","g",$2))}' \ to | awk -F= '/^result_/ { gsub(/"/,"",$2); printf("%s := %s\n",substr($1,8),$2) }' \

That worked for me as well.

[impierced]

Leseratte10, can you try compiling just an arm64 version for me to test?

[Leseratte10]

Here you go: wit_1_normal is another universal binary compiled just like 3.05a (just in case something went wrong the last time I compiled it), wit_2_arm_only is a standalone arm64 binary.

Same compiler settings / versions / SDK as before. wit_mac_test.zip

EDIT: Also, would you mind testing if the latest release of Wiimms SZS tools works on your Mac? https://szs.wiimm.de/download.html That's compiled with the same cross-compiler but a bunch of different settings, so knowing if this works or if it fails as well could help track this down.

[impierced]

Unfortunately, they all failed.

wit_1_normal: unsigned developer, was able to authenticate to allow it to run, failed to launch. wit_2_arm_only: corrupt, couldn't even authenticate to allow it to run.

Wimms SZS tools: same as wit_1_normal (latest x86_64, arm64 build).

[breakerbar]

I have an Apple silicon machine and am experiencing this issue (it manifests as "Killed: 9") and would like to help if I can.

Output (M1 MacBook Pro):

$ ./wit_1_normal
Killed: 9
$ ./wit_2_arm_only
Killed: 9
$ file *
wit_1_normal:   Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64]
wit_1_normal (for architecture x86_64): Mach-O 64-bit executable x86_64
wit_1_normal (for architecture arm64):  Mach-O 64-bit executable arm64
wit_2_arm_only: Mach-O 64-bit executable arm64
$ xattr -l *
wit_1_normal: com.apple.quarantine: 0281;6452ef7d;Firefox;FFAB83C2-B76C-4B1E-BB3B-1859E84F4989
wit_2_arm_only: com.apple.quarantine: 0281;6452ef7d;Firefox;FFAB83C2-B76C-4B1E-BB3B-1859E84F4989
$ xattr -d com.apple.quarantine *
$ xattr -l *
wit_1_normal: com.apple.provenance:
wit_2_arm_only: com.apple.provenance:
$ ./wit_1_normal
Killed: 9
$ ./wit_2_arm_only
Killed: 9

[breakerbar]

And for the szs tools (ver szs-v2.33a-r8773-mac64)

$ file bin/wbmgt
bin/wbmgt: Mach-O universal binary with 1 architecture: [x86_64:Mach-O 64-bit executable x86_64]
bin/wbmgt (for architecture x86_64):    Mach-O 64-bit executable x86_64
$ ./bin/wbmgt --help
Killed: 9
$ xattr -d com.apple.quarantine bin/wbmgt
$ ./bin/wbmgt --help
wbmgt: Wiimms BMG Tool v2.33a r8773 mac/x64 - Dirk Clemens - 2023-01-14

wbmgt : Wiimms BMG Tool : Convert and patch raw and text BMG (message) files.

Syntax: wbmgt [option]... command [option|parameter|file]...

Commands:

  VERSION    | V   : Print program name and version and exit.
  HELP       | H   : Print help for commands and options.
  CONFIG           : Show all information about the search for the
<snip>

[clunkclunk]

The SYSTEM=mac bug is solved in my working copy. I replaced line 73 of file setup.sh from | awk -F= '/^result_/ {printf("%s := %s\n",substr($1,8),gensub(/"/,"","g",$2))}' \ to | awk -F= '/^result_/ { gsub(/"/,"",$2); printf("%s := %s\n",substr($1,8),$2) }' \

That worked for me as well.

For me as well.

[flanter21]

I can compile the binaries however, i cannot link them. Anyone got any ideas? EDIT: Doesn't link when I use gcc either.

ld: warning: alignment (1) of atom '_sizeof_info_dol' from '/Users/me/Downloads/wiimms-iso-tools.source-3.05a/lib-dol.o' is too small and may result in unaligned pointers 
ld: building fixups: pointer not aligned at _SRT_UsageCount+0x1C from /Users/me/Downloads/wiimms-iso-tools.source-3.05a/dclib-numeric.o
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [gen-ui] Error 1

[evillogic]

I had a similar issue where I was getting Killed: 9 on my m3 macbook.

***    load ./share/titles.txt from http://gametdb.com/titles.txt
./load-titles.sh: line 80:  9324 Exit 3                  webget "$URI"
      9325 Killed: 9               | wit titles / - > "$DEST.tmp"
... more of the above ...
***    load ./share/titles-zhtw.txt from http://gametdb.com/titles.txt?LANG=ZHTW
./load-titles.sh: line 80:  9412 Exit 3                  webget "$URI"
      9413 Killed: 9               | wit titles / - > "$DEST.tmp"
*** install titles to /usr/local/share/wit

I solved this by running the following line after install.sh:

sudo codesign --sign - --force --preserve-metadata=entitlements,requirements,flags,runtime /usr/local/bin/wit
sudo ./load-titles.sh

It seems the latest release (v3.05a) isn't signed correctly, then load_titles fails to import titles because of this.

[unilock]

This is the crash report generated by macOS when wit (from wit-v3.05a-r8638-mac.tar.gz) is killed. It was found in Console.app after attempting to run the wit command with no arguments.

https://gist.github.com/unilock/fdc40410e52d5a2484c589044ad329c7

The codesign command in the previous comment does appear to fix the issue. (it should be executed upon every WIT-related file installed to /usr/local/bin)