search

SciresM / hactool

hactool is a tool to view information about, decrypt, and extract common file formats for the Nintendo Switch, especially Nintendo Content Archives.
ISC License
1.01k stars 155 forks source link

Building on Mac OS fails #96

Closed Pysis868 closed 4 years ago

Pysis868 commented 4 years ago

Config:

CC = gcc
CFLAGS = -O2 -Wall -Wextra -pedantic -std=gnu11 -fPIC
LDFLAGS = -lmbedtls -lmbedx509 -lmbedcrypto

Using gmake as specifically instructed.

Compiler:

gcc -v
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
Apple clang version 11.0.3 (clang-1103.0.32.59)
Target: x86_64-apple-darwin19.4.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

Linker:

ld -v
@(#)PROGRAM:ld  PROJECT:ld64-556.6
BUILD 13:10:29 Apr  7 2020
configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
LTO support using: LLVM version 11.0.3, (clang-1103.0.32.59) (static support for 26, runtime is 26)
TAPI support using: Apple TAPI version 11.0.0 (tapi-1100.0.11)

/usr/local/opt/gcc/bin/gcc-9 didn't build at all. Got stuck on stdio.h.

Building:

gmake
gmake -C mbedtls lib
gmake[1]: Entering directory '.../hactool/mbedtls'
gmake[1]: Leaving directory '.../hactool/mbedtls'
gmake hactool
gmake[1]: Entering directory '.../hactool'
gcc -I ./mbedtls/include -c -O2 -Wall -Wextra -pedantic -std=gnu11 -fPIC -D_BSD_SOURCE -D_POSIX_SOURCE -D_POSIX_C_SOURCE=200112L -D_DEFAULT_SOURCE -D__USE_MINGW_ANSI_STDIO=1 -D_FILE_OFFSET_BITS=64 -o save.o save.c
gcc -I ./mbedtls/include -c -O2 -Wall -Wextra -pedantic -std=gnu11 -fPIC -D_BSD_SOURCE -D_POSIX_SOURCE -D_POSIX_C_SOURCE=200112L -D_DEFAULT_SOURCE -D__USE_MINGW_ANSI_STDIO=1 -D_FILE_OFFSET_BITS=64 -o sha.o sha.c
...
ld: warning: ld: warning: ignoring file ./mbedtls/library/libmbedtls.a, building for macOS-x86_64 but attempting to link with file built for unknown-unsupported file format ( 0x21 0x3C 0x61 0x72 0x63 0x68 0x3E 0x0A 0x2F 0x20 0x20 0x20 0x20 0x20 0x20 0x20 )
ignoring file ./mbedtls/library/libmbedx509.a, building for macOS-x86_64 but attempting to link with file built for unknown-unsupported file format ( 0x21 0x3C 0x61 0x72 0x63 0x68 0x3E 0x0A 0x2F 0x20 0x20 0x20 0x20 0x20 0x20 0x20 )
ld: warning: ignoring file ./mbedtls/library/libmbedcrypto.a, building for macOS-x86_64 but attempting to link with file built for unknown-unsupported file format ( 0x21 0x3C 0x61 0x72 0x63 0x68 0x3E 0x0A 0x2F 0x20 0x20 0x20 0x20 0x20 0x20 0x20 )
Undefined symbols for architecture x86_64:
  "_mbedtls_cipher_cmac_finish", referenced from:
      _aes_calculate_cmac in aes.o
  "_mbedtls_cipher_cmac_starts", referenced from:
      _aes_calculate_cmac in aes.o
  "_mbedtls_cipher_cmac_update", referenced from:
      _aes_calculate_cmac in aes.o
  "_mbedtls_cipher_finish", referenced from:
      _aes_encrypt in aes.o
      _aes_decrypt in aes.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
gmake[1]: *** [Makefile:17: hactool] Error 1
gmake[1]: Leaving directory '.../hactool'
gmake: *** [Makefile:11: all] Error 2

Branch: master, origin/master Commit: f55d6f2506a38ef05ec0ab99ec4041313d5e6d7a System (OS): macOS 10.15.4

WinterMute commented 4 years ago

This builds completely fine for me OOB on Catalina with this config. Looks like you've somehow built mbedtls with the wrong architecture.

export CC=clang
export AR=ar
export CFLAGS = -O2 -Wall -Wextra -pedantic -std=gnu11 -fPIC
export LDFLAGS = -lmbedtls -lmbedx509 -lmbedcrypto

Can try this config & do a make clean_full to clean mbedtls first (once https://github.com/SciresM/hactool/pull/97 is merged).

FWIW, you can get hactool via devkitPro pacman if you're a toolchain user. i.e. sudo dkp-pacman -S hactool

Pysis868 commented 4 years ago

Commit: 2b0c791f9485f78ee359f45c91b97a48d534e9e6

> gmake clean_full
rm -f *.o hactool hactool.exe
gmake -C mbedtls clean
gmake[1]: Entering directory '.../hactool/mbedtls'
gmake[2]: Entering directory '.../hactool/mbedtls/library'
gmake[2]: Leaving directory '.../hactool/mbedtls/library'
gmake[2]: Entering directory '.../hactool/mbedtls/programs'
gmake[2]: Leaving directory '.../hactool/mbedtls/programs'
gmake[2]: Entering directory '.../hactool/mbedtls/tests'
gmake[2]: Leaving directory '.../hactool/mbedtls/tests'
gmake[1]: Leaving directory '.../hactool/mbedtls'

Config:

CC = gcc
CC = clang
AR = ar
CC1 = /usr/local/opt/gcc/bin/gcc-9
CFLAGS = -O2 -Wall -Wextra -pedantic -std=gnu11 -fPIC
LDFLAGS = -lmbedtls -lmbedx509 -lmbedcrypto

Building:

gmake
gmake -C mbedtls lib
gmake[1]: Entering directory '.../hactool/mbedtls'
gmake[2]: Entering directory '.../hactool/mbedtls/library'
  CC    aes.c
...
clang -o hactool save.o sha.o aes.o extkeys.o rsa.o npdm.o bktr.o kip.o packages.o pki.o pfs0.o hfs0.o nca0_romfs.o romfs.o utils.o nax0.o nso.o lz4.o nca.o xci.o main.o filepath.o ConvertUTF.o cJSON.o -L ./mbedtls/library -lmbedtls -lmbedx509 -lmbedcrypto
ld: warning: ignoring file ./mbedtls/library/libmbedtls.a, building for macOS-x86_64 but attempting to link with file built for unknown-unsupported file format ( 0x21 0x3C 0x61 0x72 0x63 0x68 0x3E 0x0A 0x2F 0x20 0x20 0x20 0x20 0x20 0x20 0x20 )
ld: warning: ignoring file ./mbedtls/library/libmbedcrypto.a, building for macOS-x86_64 but attempting to link with file built for unknown-unsupported file format ( 0x21 0x3C 0x61 0x72 0x63 0x68 0x3E 0x0A 0x2F 0x20 0x20 0x20 0x20 0x20 0x20 0x20 )
ld: warning: ignoring file ./mbedtls/library/libmbedx509.a, building for macOS-x86_64 but attempting to link with file built for unknown-unsupported file format ( 0x21 0x3C 0x61 0x72 0x63 0x68 0x3E 0x0A 0x2F 0x20 0x20 0x20 0x20 0x20 0x20 0x20 )
Undefined symbols for architecture x86_64:
...
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
gmake[1]: *** [Makefile:17: hactool] Error 1
gmake[1]: Leaving directory '.../hactool'
gmake: *** [Makefile:11: all] Error 2
WinterMute commented 4 years ago

Why are you adding extra bits to the config I posted in https://github.com/SciresM/hactool/issues/96#issuecomment-653614711?

This builds fine OOB on a fresh Catalina install with xcode command line tools. Would personally recommend staying far, far away from brew or whatever that is you're using. It's clearly broken.

Pysis868 commented 4 years ago

Just how I keep old entries in case I want to switch them on again quickly. Tried it with CC1 = gcc as well to disable that part.

Xcode: 11.4.1

> ls -l (which gmake)
lrwxr-xr-x 1 ... admin 28 2020-04-27 15:03 /usr/local/bin/gmake -> ../Cellar/make/4.3/bin/gmake

> brew info make
make: stable 4.3 (bottled)
Utility for directing compilation
https://www.gnu.org/software/make/
/usr/local/Cellar/make/4.3 (15 files, 1MB) *
  Poured from bottle on 2020-04-27 at 15:03:37
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/make.rb
...

> brew info automake
automake: stable 1.16.2 (bottled)
Tool for generating GNU Standards-compliant Makefiles
https://www.gnu.org/software/automake/
/usr/local/Cellar/automake/1.16.2 (131 files, 3.4MB) *
  Poured from bottle on 2020-04-27 at 15:10:20
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/automake.rb
...

> brew info autoconf
autoconf: stable 2.69 (bottled)
Automatic configure script builder
https://www.gnu.org/software/autoconf
/usr/local/Cellar/autoconf/2.69 (70 files, 3.0MB) *
  Poured from bottle on 2017-08-06 at 14:00:31
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/autoconf.rb
...
WinterMute commented 4 years ago

k. One last shot.

Try exactly this config as written. No modifications of any kind.

export CC=/usr/bin/clang
export AR=/usr/bin/ar
export CFLAGS = -O2 -Wall -Wextra -pedantic -std=gnu11 -fPIC
export LDFLAGS = -lmbedtls -lmbedx509 -lmbedcrypto

and then run /usr/bin/make directly.

Whatever you're trying to compile hactool with is broken. If using direct paths to official apple tools doesn't work then your machine is hosed and I'd reinstall your OS.

Pysis868 commented 4 years ago

That seemed to do it, thanks.

Besides trying to keep the config.mk file clean, I noticed the ar is different, but I was also using gmake earlier based on the readme mention. Used regular apple's make this time.

~/D/s/hactool (master=) gmake clean_full
rm -f *.o hactool hactool.exe
gmake -C mbedtls clean
gmake[1]: Entering directory '.../hactool/mbedtls'
gmake[2]: Entering directory '.../hactool/mbedtls/library'
gmake[2]: Leaving directory '.../hactool/mbedtls/library'
gmake[2]: Entering directory '.../hactool/mbedtls/programs'
gmake[2]: Leaving directory '.../hactool/mbedtls/programs'
gmake[2]: Entering directory '.../hactool/mbedtls/tests'
gmake[2]: Leaving directory '.../hactool/mbedtls/tests'
gmake[1]: Leaving directory '.../hactool/mbedtls'
~/D/s/hactool (master=) which make
/usr/bin/make
~/D/s/hactool (master=) which clang
/usr/bin/clang
~/D/s/hactool (master=) which ar
/usr/local/bin/ar
~/D/s/hactool (master=) pbpaste > config.mk
~/D/s/hactool (master=) cat config.mk
export CC=/usr/bin/clang
export AR=/usr/bin/ar
export CFLAGS = -O2 -Wall -Wextra -pedantic -std=gnu11 -fPIC
export LDFLAGS = -lmbedtls -lmbedx509 -lmbedcrypto
~/D/s/hactool (master=) ls -l hactool
-rwxr-xr-x 1 ... staff 473K 2020-07-03 16:54 hactool

If I try from the repos, I get this:

~ sudo dkp-pacman -S hactool
:: Synchronizing package databases...
 dkp-libs is up to date
 dkp-osx is up to date
resolving dependencies...
looking for conflicting packages...

Packages (1) hactool-1.3.2-1
...
(1/1) checking available disk space                                         [###########################################] 100%
error: Partition / is mounted read only
WinterMute commented 4 years ago

fwiw apple make is gnu make.

 % make --version
GNU Make 3.81
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for i386-apple-darwin11.3.0
WinterMute commented 4 years ago

If I try from the repos, I get this:

~ sudo dkp-pacman -S hactool
:: Synchronizing package databases...
 dkp-libs is up to date
 dkp-osx is up to date
resolving dependencies...
looking for conflicting packages...

Packages (1) hactool-1.3.2-1
...
(1/1) checking available disk space                                         [###########################################] 100%
error: Partition / is mounted read only

That's the issue with Catalina https://github.com/devkitPro/pacman/issues/15

It's been fixed with latest pacman installer https://github.com/devkitPro/pacman/releases/latest