OSX compilation broken with "Add support for compilation on RaspberryPiZero (armv6)" commit

[hduarte]

The commit Add support for compilation on RaspberryPiZero (armv6) added the following options to the linker, on the makefile of the client component:

• -Wl,--as-needed -latomic -Wl,--no-as-needed

The --as-needed and --no-as-needed are not support by OSX linker resulting in the following error:

...
[=] GEN lualibs/mf_default_keys.lua
[=] LD proxmark3
ld: unknown option: --as-needed
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [proxmark3] Error 1
make: *** [client/all] Error 2
...

My linker version:

 ~ ld -v                                                                                                                               
@(#)PROGRAM:ld  PROJECT:ld64-450.3
BUILD 18:45:16 Apr  4 2019
configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
LTO support using: LLVM version 10.0.1, (clang-1001.0.46.4) (static support for 22, runtime is 22)
TAPI support using: Apple TAPI version 10.0.1 (tapi-1001.0.4.1)

brew config:

==> Configuration
HOMEBREW_VERSION: 2.1.6
ORIGIN: https://github.com/Homebrew/brew
HEAD: fddf5b16bd8df577380a1d810d17825ef611b81f
Last commit: 2 weeks ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 82ae7037bd677d2591bb99a7103f7c3da90a7ece
Core tap last commit: 8 hours ago
HOMEBREW_PREFIX: /usr/local
HOMEBREW_NO_ANALYTICS_THIS_RUN: 1
CPU: octa-core 64-bit kabylake
Homebrew Ruby: 2.3.7 => /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/bin/ruby
Clang: 10.0 build 1001
Git: 2.20.1 => /Applications/Xcode.app/Contents/Developer/usr/bin/git
Curl: 7.54.0 => /usr/bin/curl
macOS: 10.14.5-x86_64
CLT: 11.0.0.0.1.1559496560
Xcode: 10.2.1
CLT headers: 11.0.0.0.1.1559496560

[iceman1001]

@doegox bugger... Have to make it conditional to be excluded on OS X...

[doegox]

I don't get it... @hduarte you're compiling with clang instead of gcc ?

[hduarte]

@doegox yes clang. By using the brew formula for proxmark this is the default behaviour on osx.

[doegox]

ok so best is to detect when clang is used rather than detecting when OSX is used I guess. I'll look into it...

[hduarte]

Thks @doegox let me know if you need any help

[doegox]

Well @hduarte I just realized I could compile with clang under linux as well by setting it explicitly in the Makefile, and I don't have any error if I define LD=clang++. So please could you test locally the following makefile changes:

diff --git a/client/Makefile b/client/Makefile
index 135b143e..e1c8b818 100644
--- a/client/Makefile
+++ b/client/Makefile
@@ -15,9 +15,9 @@ ifneq ($(V),1)
 endif
 # To see full command lines, use make V=1

-CC = gcc
-CXX = g++
-LD = g++
+CC = clang
+CXX = clang++
+LD = clang++
 TAR = tar
 TARFLAGS = -C .. --ignore-failed-read -rvf
 RM = rm -f

[hduarte]

@doegox It still doesn't work :/

[doegox]

Can you dump the relevant lines of the compilation with a make V=1 client ? Thanks!

[hduarte]

make V=1 client
===================================================================
Platform name:     Proxmark3 rdv4
PLATFORM:          PM3RDV4
Platform extras:   BTADDON
Included options:  SMARTCARD FLASH FPC_USART_HOST LF HITAG ISO15693 LEGICRF ISO14443b ISO14443a ICLASS FELICA HFSNIFF FPC_USART
Standalone mode:   LF_SAMYRUN
===================================================================
[*] MAKE client/all
/Applications/Xcode.app/Contents/Developer/usr/bin/make --no-print-directory -C client all
[*] MAKE liblua for macosx
/Applications/Xcode.app/Contents/Developer/usr/bin/make --no-print-directory -C ../liblua macosx
/Applications/Xcode.app/Contents/Developer/usr/bin/make all SYSCFLAGS="-DLUA_USE_MACOSX" SYSLIBS="-lreadline"
make[3]: Nothing to be done for `all'.
[*] MAKE jansson
/Applications/Xcode.app/Contents/Developer/usr/bin/make --no-print-directory -C ./jansson all
make[2]: Nothing to be done for `all'.
[*] MAKE mbedtls
/Applications/Xcode.app/Contents/Developer/usr/bin/make --no-print-directory -C ../common/mbedtls all
make[2]: Nothing to be done for `all'.
[*] MAKE tinycbor
/Applications/Xcode.app/Contents/Developer/usr/bin/make --no-print-directory -C ./tinycbor all
make[2]: Nothing to be done for `all'.
[=] LD proxmark3
clang++  -framework Foundation -framework AppKit obj/proxmark3.o obj/uart_posix.o obj/uart_win32.o obj/ui.o obj/commonutil.o obj/util.o obj/util_posix.o obj/scandir.o obj/crc16.o obj/comms.o obj/crapto1/crapto1.o obj/crapto1/crypto1.o obj/mifare/mfkey.o obj/tea.o obj/fido/additional_ca.o obj/fido/cose.o obj/fido/cbortools.o obj/fido/fidocore.o obj/crypto/asn1dump.o obj/crypto/libpcrypto.o obj/crypto/asn1utils.o obj/cliparser/argtable3.o obj/cliparser/cliparser.o obj/loclass/cipher.o obj/loclass/cipherutils.o obj/loclass/ikeys.o obj/loclass/elite_crack.o obj/loclass/fileutils.o obj/whereami.o obj/mifare/mifarehost.o obj/parity.o obj/crc.o obj/crc64.o obj/legic_prng.o obj/iso15693tools.o obj/prng.o obj/graph.o obj/cmddata.o obj/lfdemod.o obj/emv/crypto_polarssl.o obj/emv/crypto.o obj/emv/emv_pk.o obj/emv/emv_pki.o obj/emv/emv_pki_priv.o obj/emv/test/cryptotest.o obj/emv/apduinfo.o obj/emv/dump.o obj/emv/tlv.o obj/emv/emv_tags.o obj/emv/dol.o obj/emv/emvjson.o obj/emv/emvcore.o obj/emv/test/crypto_test.o obj/emv/test/sda_test.o obj/emv/test/dda_test.o obj/emv/test/cda_test.o obj/emv/cmdemv.o obj/emv/emv_roca.o obj/mifare/mifare4.o obj/mifare/mad.o obj/mifare/ndef.o obj/cmdanalyse.o obj/cmdhf.o obj/cmdhflist.o obj/cmdhf14a.o obj/cmdhf14b.o obj/cmdhf15.o obj/cmdhfepa.o obj/cmdhflegic.o obj/cmdhficlass.o obj/cmdhfmf.o obj/cmdhfmfu.o obj/cmdhfmfp.o obj/cmdhfmfhard.o obj/hardnested/hardnested_bruteforce.o obj/cmdhfmfdes.o obj/cmdhftopaz.o obj/cmdhffido.o obj/cmdhffelica.o obj/cmdhw.o obj/cmdlf.o obj/cmdlfawid.o obj/cmdlfcotag.o obj/cmdlfem4x.o obj/cmdlffdx.o obj/cmdlfguard.o obj/cmdlfhid.o obj/cmdlfhitag.o obj/cmdlfio.o obj/cmdlfindala.o obj/cmdlfjablotron.o obj/cmdlfkeri.o obj/cmdlfnexwatch.o obj/cmdlfnedap.o obj/cmdlfnoralsy.o obj/cmdlfpac.o obj/cmdlfparadox.o obj/cmdlfpcf7931.o obj/cmdlfpresco.o obj/cmdlfpyramid.o obj/cmdlfsecurakey.o obj/cmdlft55xx.o obj/cmdlfti.o obj/cmdlfviking.o obj/cmdlfvisa2000.o obj/cmdtrace.o obj/cmdflashmem.o obj/cmdsmartcard.o obj/cmdusart.o obj/cmdparser.o obj/cmdmain.o obj/pm3_binlib.o obj/scripting.o obj/cmdscript.o obj/pm3_bitlib.o obj/protocols.o obj/cmdcrc.o obj/reveng/preset.o obj/reveng/reveng.o obj/reveng/cli.o obj/reveng/bmpbit.o obj/reveng/model.o obj/reveng/poly.o obj/bucketsort.o obj/util_darwin.o obj/guidummy.o obj/hardnested/hardnested_bf_core_NOSIMD.o obj/hardnested/hardnested_bitarray_core_NOSIMD.o obj/hardnested/hardnested_bf_core_MMX.o obj/hardnested/hardnested_bitarray_core_MMX.o obj/hardnested/hardnested_bf_core_SSE2.o obj/hardnested/hardnested_bitarray_core_SSE2.o obj/hardnested/hardnested_bf_core_AVX.o obj/hardnested/hardnested_bitarray_core_AVX.o obj/hardnested/hardnested_bf_core_AVX2.o obj/hardnested/hardnested_bitarray_core_AVX2.o obj/hardnested/hardnested_bf_core_AVX512.o obj/hardnested/hardnested_bitarray_core_AVX512.o obj/deflate.o obj/adler32.o obj/trees.o obj/zutil.o obj/inflate.o obj/inffast.o obj/inftrees.o -L/usr/local/opt/readline/lib -L/opt/local/lib -L/usr/local/lib -lreadline -lpthread -lm -Wl,--as-needed -latomic -Wl,--no-as-needed ../liblua/liblua.a ./jansson/libjansson.a ../common/mbedtls/libmbedtls.a ./tinycbor/tinycbor.a   -o proxmark3
ld: warning: directory not found for option '-L/opt/local/lib'
ld: unknown option: --as-needed
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [proxmark3] Error 1
make: *** [client/all] Error 2

Clang/Clang++ version

/usr/bin/clang++ --version                                                                                                                                                                                         

Apple LLVM version 10.0.1 (clang-1001.0.46.4)
Target: x86_64-apple-darwin18.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

And I'm using the latest Xcode 10.2.1 with the latest command line tools.

[doegox]

We need to see which ld is called by clang, could you add -v to the LDLIBS ? Then last big line will be the ld invocation.

[hduarte]

Here is the output

...
[=] GEN lualibs/mf_default_keys.lua
awk -f default_keys_dic2lua.awk default_keys.dic > lualibs/mf_default_keys.lua
[=] LD proxmark3
clang++  -framework Foundation -framework AppKit obj/proxmark3.o obj/uart_posix.o obj/uart_win32.o obj/ui.o obj/commonutil.o obj/util.o obj/util_posix.o obj/scandir.o obj/crc16.o obj/comms.o obj/crapto1/crapto1.o obj/crapto1/crypto1.o obj/mifare/mfkey.o obj/tea.o obj/fido/additional_ca.o obj/fido/cose.o obj/fido/cbortools.o obj/fido/fidocore.o obj/crypto/asn1dump.o obj/crypto/libpcrypto.o obj/crypto/asn1utils.o obj/cliparser/argtable3.o obj/cliparser/cliparser.o obj/loclass/cipher.o obj/loclass/cipherutils.o obj/loclass/ikeys.o obj/loclass/elite_crack.o obj/loclass/fileutils.o obj/whereami.o obj/mifare/mifarehost.o obj/parity.o obj/crc.o obj/crc64.o obj/legic_prng.o obj/iso15693tools.o obj/prng.o obj/graph.o obj/cmddata.o obj/lfdemod.o obj/emv/crypto_polarssl.o obj/emv/crypto.o obj/emv/emv_pk.o obj/emv/emv_pki.o obj/emv/emv_pki_priv.o obj/emv/test/cryptotest.o obj/emv/apduinfo.o obj/emv/dump.o obj/emv/tlv.o obj/emv/emv_tags.o obj/emv/dol.o obj/emv/emvjson.o obj/emv/emvcore.o obj/emv/test/crypto_test.o obj/emv/test/sda_test.o obj/emv/test/dda_test.o obj/emv/test/cda_test.o obj/emv/cmdemv.o obj/emv/emv_roca.o obj/mifare/mifare4.o obj/mifare/mad.o obj/mifare/ndef.o obj/cmdanalyse.o obj/cmdhf.o obj/cmdhflist.o obj/cmdhf14a.o obj/cmdhf14b.o obj/cmdhf15.o obj/cmdhfepa.o obj/cmdhflegic.o obj/cmdhficlass.o obj/cmdhfmf.o obj/cmdhfmfu.o obj/cmdhfmfp.o obj/cmdhfmfhard.o obj/hardnested/hardnested_bruteforce.o obj/cmdhfmfdes.o obj/cmdhftopaz.o obj/cmdhffido.o obj/cmdhffelica.o obj/cmdhw.o obj/cmdlf.o obj/cmdlfawid.o obj/cmdlfcotag.o obj/cmdlfem4x.o obj/cmdlffdx.o obj/cmdlfguard.o obj/cmdlfhid.o obj/cmdlfhitag.o obj/cmdlfio.o obj/cmdlfindala.o obj/cmdlfjablotron.o obj/cmdlfkeri.o obj/cmdlfnexwatch.o obj/cmdlfnedap.o obj/cmdlfnoralsy.o obj/cmdlfpac.o obj/cmdlfparadox.o obj/cmdlfpcf7931.o obj/cmdlfpresco.o obj/cmdlfpyramid.o obj/cmdlfsecurakey.o obj/cmdlft55xx.o obj/cmdlfti.o obj/cmdlfviking.o obj/cmdlfvisa2000.o obj/cmdtrace.o obj/cmdflashmem.o obj/cmdsmartcard.o obj/cmdusart.o obj/cmdparser.o obj/cmdmain.o obj/pm3_binlib.o obj/scripting.o obj/cmdscript.o obj/pm3_bitlib.o obj/protocols.o obj/cmdcrc.o obj/reveng/preset.o obj/reveng/reveng.o obj/reveng/cli.o obj/reveng/bmpbit.o obj/reveng/model.o obj/reveng/poly.o obj/bucketsort.o obj/util_darwin.o obj/guidummy.o obj/hardnested/hardnested_bf_core_NOSIMD.o obj/hardnested/hardnested_bitarray_core_NOSIMD.o obj/hardnested/hardnested_bf_core_MMX.o obj/hardnested/hardnested_bitarray_core_MMX.o obj/hardnested/hardnested_bf_core_SSE2.o obj/hardnested/hardnested_bitarray_core_SSE2.o obj/hardnested/hardnested_bf_core_AVX.o obj/hardnested/hardnested_bitarray_core_AVX.o obj/hardnested/hardnested_bf_core_AVX2.o obj/hardnested/hardnested_bitarray_core_AVX2.o obj/hardnested/hardnested_bf_core_AVX512.o obj/hardnested/hardnested_bitarray_core_AVX512.o obj/deflate.o obj/adler32.o obj/trees.o obj/zutil.o obj/inflate.o obj/inffast.o obj/inftrees.o -L/usr/local/opt/readline/lib -L/opt/local/lib -L/usr/local/lib -lreadline -lpthread -lm -Wl,--as-needed -latomic -Wl,--no-as-needed -v ../liblua/liblua.a ./jansson/libjansson.a ../common/mbedtls/libmbedtls.a ./tinycbor/tinycbor.a   -o proxmark3
Apple LLVM version 10.0.1 (clang-1001.0.46.4)
Target: x86_64-apple-darwin18.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
 "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld" -demangle -lto_library /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/libLTO.dylib -dynamic -arch x86_64 -macosx_version_min 10.14.0 -syslibroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk -o proxmark3 -L/usr/local/opt/readline/lib -L/opt/local/lib -L/usr/local/lib -framework Foundation -framework AppKit obj/proxmark3.o obj/uart_posix.o obj/uart_win32.o obj/ui.o obj/commonutil.o obj/util.o obj/util_posix.o obj/scandir.o obj/crc16.o obj/comms.o obj/crapto1/crapto1.o obj/crapto1/crypto1.o obj/mifare/mfkey.o obj/tea.o obj/fido/additional_ca.o obj/fido/cose.o obj/fido/cbortools.o obj/fido/fidocore.o obj/crypto/asn1dump.o obj/crypto/libpcrypto.o obj/crypto/asn1utils.o obj/cliparser/argtable3.o obj/cliparser/cliparser.o obj/loclass/cipher.o obj/loclass/cipherutils.o obj/loclass/ikeys.o obj/loclass/elite_crack.o obj/loclass/fileutils.o obj/whereami.o obj/mifare/mifarehost.o obj/parity.o obj/crc.o obj/crc64.o obj/legic_prng.o obj/iso15693tools.o obj/prng.o obj/graph.o obj/cmddata.o obj/lfdemod.o obj/emv/crypto_polarssl.o obj/emv/crypto.o obj/emv/emv_pk.o obj/emv/emv_pki.o obj/emv/emv_pki_priv.o obj/emv/test/cryptotest.o obj/emv/apduinfo.o obj/emv/dump.o obj/emv/tlv.o obj/emv/emv_tags.o obj/emv/dol.o obj/emv/emvjson.o obj/emv/emvcore.o obj/emv/test/crypto_test.o obj/emv/test/sda_test.o obj/emv/test/dda_test.o obj/emv/test/cda_test.o obj/emv/cmdemv.o obj/emv/emv_roca.o obj/mifare/mifare4.o obj/mifare/mad.o obj/mifare/ndef.o obj/cmdanalyse.o obj/cmdhf.o obj/cmdhflist.o obj/cmdhf14a.o obj/cmdhf14b.o obj/cmdhf15.o obj/cmdhfepa.o obj/cmdhflegic.o obj/cmdhficlass.o obj/cmdhfmf.o obj/cmdhfmfu.o obj/cmdhfmfp.o obj/cmdhfmfhard.o obj/hardnested/hardnested_bruteforce.o obj/cmdhfmfdes.o obj/cmdhftopaz.o obj/cmdhffido.o obj/cmdhffelica.o obj/cmdhw.o obj/cmdlf.o obj/cmdlfawid.o obj/cmdlfcotag.o obj/cmdlfem4x.o obj/cmdlffdx.o obj/cmdlfguard.o obj/cmdlfhid.o obj/cmdlfhitag.o obj/cmdlfio.o obj/cmdlfindala.o obj/cmdlfjablotron.o obj/cmdlfkeri.o obj/cmdlfnexwatch.o obj/cmdlfnedap.o obj/cmdlfnoralsy.o obj/cmdlfpac.o obj/cmdlfparadox.o obj/cmdlfpcf7931.o obj/cmdlfpresco.o obj/cmdlfpyramid.o obj/cmdlfsecurakey.o obj/cmdlft55xx.o obj/cmdlfti.o obj/cmdlfviking.o obj/cmdlfvisa2000.o obj/cmdtrace.o obj/cmdflashmem.o obj/cmdsmartcard.o obj/cmdusart.o obj/cmdparser.o obj/cmdmain.o obj/pm3_binlib.o obj/scripting.o obj/cmdscript.o obj/pm3_bitlib.o obj/protocols.o obj/cmdcrc.o obj/reveng/preset.o obj/reveng/reveng.o obj/reveng/cli.o obj/reveng/bmpbit.o obj/reveng/model.o obj/reveng/poly.o obj/bucketsort.o obj/util_darwin.o obj/guidummy.o obj/hardnested/hardnested_bf_core_NOSIMD.o obj/hardnested/hardnested_bitarray_core_NOSIMD.o obj/hardnested/hardnested_bf_core_MMX.o obj/hardnested/hardnested_bitarray_core_MMX.o obj/hardnested/hardnested_bf_core_SSE2.o obj/hardnested/hardnested_bitarray_core_SSE2.o obj/hardnested/hardnested_bf_core_AVX.o obj/hardnested/hardnested_bitarray_core_AVX.o obj/hardnested/hardnested_bf_core_AVX2.o obj/hardnested/hardnested_bitarray_core_AVX2.o obj/hardnested/hardnested_bf_core_AVX512.o obj/hardnested/hardnested_bitarray_core_AVX512.o obj/deflate.o obj/adler32.o obj/trees.o obj/zutil.o obj/inflate.o obj/inffast.o obj/inftrees.o -lreadline -lpthread -lm --as-needed -latomic --no-as-needed ../liblua/liblua.a ./jansson/libjansson.a ../common/mbedtls/libmbedtls.a ./tinycbor/tinycbor.a -L/usr/local/lib -lc++ -lSystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/10.0.1/lib/darwin/libclang_rt.osx.a
ld: warning: directory not found for option '-L/opt/local/lib'
ld: unknown option: --as-needed
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [proxmark3] Error 1
make: *** [client/all] Error 2

[doegox]

Would this one work?

diff --git a/client/Makefile b/client/Makefile
index 135b143e..5b7cc9d4 100644
--- a/client/Makefile
+++ b/client/Makefile
@@ -31,7 +31,11 @@ platform = $(shell uname)
 VPATH = ../common ../zlib ../uart
 OBJDIR = obj

-LDLIBS = -L/opt/local/lib -L/usr/local/lib -lreadline -lpthread -lm -Wl,--as-needed -latomic -Wl,--no-as-needed
+LDLIBS = -L/opt/local/lib -L/usr/local/lib -lreadline -lpthread -lm
+UNAME_S := $(shell uname -s)
+ifneq ($(UNAME_S),Darwin)
+    LDLIBS += -Wl,--as-needed -latomic -Wl,--no-as-needed
+endif
 LUALIB = ../liblua/liblua.a
 JANSSONLIBPATH = ./jansson
 JANSSONLIB = $(JANSSONLIBPATH)/libjansson.a

@iceman1001 is the uname -s working under Proxspace and WSL ?

[iceman1001]

the -s doesn't do anything on WSL or Proxspace. You already have the platform variable a few lines above, no need to introduce uname_s variable.

It compiles on WSL, proxspace..

Maybe check for Darwin like this?

ifneq (,$(findstring Darwin,$(platform)))
    LDLIBS += -Wl,--as-needed -latomic -Wl,--no-as-needed
endif

[doegox]

Oops I missed we've already $(platform) Under linux it's Linux so if it's Darwin under MacOSX, no need for findstring.

ifneq ($(platform),Darwin)
    LDLIBS += -Wl,--as-needed -latomic -Wl,--no-as-needed
endif

[doegox]

ok we've already that syntax later in the makefle, I pushed the fix

[iceman1001]

maybe @hduarte should report back first before closing?

[hduarte]

@doegox @iceman1001 all good now. Thks for the help and fix <3 !

[doegox]

Ha sorry, my commit message closed it automatically

[hduarte]

No problem :)