Closed benmkw closed 3 years ago
michaeleisel
commented
3 years ago Yes... arm mac... 😅
So abseil, one of its dependencies, is failing to build. You can see in the Makefile where we build it with cmake/make. I don't have an arm mac, but if you can figure out how to tweak that bit, then we can solve this particular build failure.
benmkw
commented
3 years ago hm so it seems abseil builds fine
$ lipo -info /Users/benediktmandelkow/gits/forks/zld/abseil-cpp-20200225/build/libabsl.a
Non-fat file: /Users/benediktmandelkow/gits/forks/zld/abseil-cpp-20200225/build/libabsl.a is architecture: arm64the problem is this invocation
/usr/bin/clang -x c++ -target x86_64-apple-macos10.14 -fmessage-length\=150 -fdiagnostics-show-note-include-stack -fmacro-backtrace-limit\=0 -fcolor-diagnostics -std\=c++1z -stdlib\=libc++ -Wno-trigraphs -fpascal-strings -O3 -fno-common -mdynamic-no-pic -Wno-missing-field-initializers -Wmissing-prototypes -Wunreachable-code -Wno-non-virtual-dtor -Woverloaded-virtual -Wno-exit-time-destructors -Wmissing-braces -Wparentheses -Wswitch -Wunused-function -Wunused-label -Wno-unused-parameter -Wunused-variable -Wunused-value -Wempty-body -Wno-uninitialized -Wunknown-pragmas -Wno-shadow -Wfour-char-constants -Wno-conversion -Wconstant-conversion -Wint-conversion -Wbool-conversion -Wenum-conversion -Wno-float-conversion -Wno-non-literal-null-conversion -Wno-objc-literal-conversion -Wsign-compare -Wno-shorten-64-to-32 -Wnewline-eof -Wno-c++11-extensions -DLD_VERS\=\"ld64-\" -DNDEBUG -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.0.sdk -fasm-blocks -fstrict-aliasing -Wdeprecated-declarations -Wno-invalid-offsetof -g -msse4.2 -Wno-sign-conversion -Winfinite-recursion -Wmove -Wno-comma -Wno-block-capture-autoreleasing -Wno-strict-prototypes -Wno-range-loop-analysis -Wno-semicolon-before-method-body -iquote /Users/benediktmandelkow/gits/forks/zld/build/Build/Intermediates.noindex/zld.build/Release/zld.build/zld-generated-files.hmap -I/Users/benediktmandelkow/gits/forks/zld/build/Build/Intermediates.noindex/zld.build/Release/zld.build/zld-own-target-headers.hmap -I/Users/benediktmandelkow/gits/forks/zld/build/Build/Intermediates.noindex/zld.build/Release/zld.build/zld-all-target-headers.hmap -iquote /Users/benediktmandelkow/gits/forks/zld/build/Build/Intermediates.noindex/zld.build/Release/zld.build/zld-project-headers.hmap -I/Users/benediktmandelkow/gits/forks/zld/build/Build/Products/Release/include -I/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/local/include -I/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/local/include -I/Users/benediktmandelkow/gits/forks/zld/tapi-1100.0.11/include -I/Users/benediktmandelkow/gits/forks/zld/dyld-733.6/include -I/Users/benediktmandelkow/gits/forks/zld/llvm-8.0.1.src/include -I/Users/benediktmandelkow/gits/forks/zld/cfe-8.0.1.src/include -I/Users/benediktmandelkow/gits/forks/zld/abseil-cpp-20200225 -I/Users/benediktmandelkow/gits/forks/zld/pstl/include -I/Users/benediktmandelkow/gits/forks/zld/pstl/stdlib -I/Users/benediktmandelkow/gits/forks/zld/tbb/include -I/Users/benediktmandelkow/gits/forks/zld/build/Build/Intermediates.noindex/zld.build/Release/zld.build/DerivedSources-normal/x86_64 -I/Users/benediktmandelkow/gits/forks/zld/build/Build/Intermediates.noindex/zld.build/Release/zld.build/DerivedSources/x86_64 -I/Users/benediktmandelkow/gits/forks/zld/build/Build/Intermediates.noindex/zld.build/Release/zld.build/DerivedSources -Wall -F/Users/benediktmandelkow/gits/forks/zld/build/Build/Products/Release -MMD -MT dependencies -MF /Users/benediktmandelkow/gits/forks/zld/build/Build/Intermediates.noindex/zld.build/Release/zld.build/Objects-normal/x86_64/stubs.d --serialize-diagnostics /Users/benediktmandelkow/gits/forks/zld/build/Build/Intermediates.noindex/zld.build/Release/zld.build/Objects-normal/x86_64/stubs.dia -c /Users/benediktmandelkow/gits/forks/zld/ld/src/ld/passes/stubs/stubs.cpp -o /Users/benediktmandelkow/gits/forks/zld/build/Build/Intermediates.noindex/zld.build/Release/zld.build/Objects-normal/x86_64/stubs.onotably /usr/bin/clang -x c++ -target x86_64-apple-macos10.14 which seems to come from xcodebuild -project ld/zld.xcodeproj -scheme zld -derivedDataPath build -configuration Release build
I tried xcodebuild -project ld/zld.xcodeproj -scheme zld -derivedDataPath build -configuration Release build -target arm64 which results in xcodebuild: error: You cannot specify both a scheme and targets. and -arch arm64 which gives or updating it to include at least one value from VALID_ARCHS (x86_64, i386, ppc). (in target 'zld' from project 'zld').
With this hacky diff
diff --git a/ld/zld.xcodeproj/project.pbxproj b/ld/zld.xcodeproj/project.pbxproj
index 40f5dc1..e4b3a8d 100644
--- a/ld/zld.xcodeproj/project.pbxproj
+++ b/ld/zld.xcodeproj/project.pbxproj
@@ -1308,6 +1308,7 @@
SECTORDER_FLAGS = "";
STRIP_INSTALLED_PRODUCT = NO;
TOOLCHAINS = osx;
+ VALID_ARCHS = arm64;
VERSIONING_SYSTEM = "apple-generic";
WARNING_CFLAGS = "-Wall";
};
@@ -1401,7 +1402,7 @@
STRIP_INSTALLED_PRODUCT = NO;
STRIP_STYLE = debugging;
TOOLCHAINS = osx;
- VALID_ARCHS = "x86_64 i386 ppc";
+ VALID_ARCHS = arm64;
VERSIONING_SYSTEM = "apple-generic";
WARNING_CFLAGS = "-Wall";
};
@@ -1755,7 +1756,7 @@
STRIP_INSTALLED_PRODUCT = NO;
STRIP_STYLE = debugging;
TOOLCHAINS = osx;
- VALID_ARCHS = "x86_64 i386 ppc";
+ VALID_ARCHS = arm64;
VERSIONING_SYSTEM = "apple-generic";
WARNING_CFLAGS = "-Wall";
};
@@ -1776,7 +1777,7 @@
PRODUCT_NAME = rebase;
STRIP_INSTALLED_PRODUCT = YES;
STRIP_STYLE = debugging;
- VALID_ARCHS = "i386 ppc x86_64";
+ VALID_ARCHS = arm64;
};
name = "Release-assert";
};
@@ -2128,6 +2129,7 @@
INSTALL_PATH = "$(DT_VARIANT)/$(TOOLCHAIN_INSTALL_DIR)/usr/bin";
PREBINDING = NO;
PRODUCT_NAME = rebase;
+ VALID_ARCHS = arm64;
};
name = Debug;
};
@@ -2146,7 +2148,7 @@
PRODUCT_NAME = rebase;
STRIP_INSTALLED_PRODUCT = YES;
STRIP_STYLE = debugging;
- VALID_ARCHS = "i386 ppc x86_64";
+ VALID_ARCHS = arm64;
};
name = Release;
};I run into error: The linked library 'libtbb.a' is missing one or more architectures required by this target: arm64. (in target 'zld' from project 'zld') which I can't really explain yet :D
michaeleisel
commented
3 years ago In that case, it's not abseil, but tbb, another dependency you can see in the Makefile
benmkw
commented
3 years ago ah they (libmimalloc.a and libtbb.a) were just build for x86 and committed in https://github.com/michaeleisel/zld/pull/18 and not build from source
thii
commented
3 years ago @michaeleisel How did you build libmimalloc.a? Do you mind if I build and submit those arm binaries?
michaeleisel
commented
3 years ago mimalloc? why do you need to build mimalloc? (i played with it at one point but i can't see any trace of it left in master). and i'd be happy to merge any prs to support arm
thii
commented
3 years ago ah, it's not being used, nevermind
benmkw
commented
3 years ago Do you mind if I build and submit those arm binaries?
it would be best to integrate whats done https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/tbb.rb into the makefile or was there a specific reason why thats not possible? Of course its tedious work :/ There is also at least one issue/ pr open at tdd about issues on arm so it would probably need to be updated again if it even works at all already (maybe the parts are not used here)
i can't see any trace of it left in master
Its still in ld/libmimalloc.a
benmkw
commented
3 years ago tbb can be build with the homebrew formula but there are are a bunch of x86 simd intrinsics issues as well, pstl seems to be more of an intel thing so this just wont work I guess 😄
michaeleisel
commented
3 years ago there may be larger issues too. for example, i was told that the arm linker signs while linking, which the x86 one doesn't do. but i may make a version without these dependencies and we can see how it goes
benmkw
commented
3 years ago The signing isn't that bad, several projects like zig and go already solved that and I think it's just computing a relatively basic hash.
michaeleisel
commented
3 years ago Reopening this issue to serve as a catch-all for ARM stuff, since it has the most context in its comments
michaeleisel
commented
3 years ago By the way, how are things with Rosetta code? I'll start putting up releases with pre-compiled executables for now in case it helps: https://github.com/michaeleisel/zld/releases/download/1.2.1/zld.tar.xz
michaeleisel
commented
3 years ago As for tbb, currently we download a pre-compiled version of it, and that version doesn't have arm (nor does their most recent version for mac). @benmkw can you try building it from scratch on your arm mac? I was able to cross-compile it for arm for iOS (cmake -DCMAKE_OSX_ARCHITECTURES=arm64 -DTBB_TEST=OFF -DCMAKE_SYSTEM_PROCESSOR=arm -DCMAKE_THREAD_LIBS_INIT="-lpthread" -DCMAKE_HAVE_THREADS_LIBRARY=1 -DCMAKE_USE_WIN32_THREADS_INIT=0 -DCMAKE_USE_PTHREADS_INIT=1 -DCMAKE_OSX_SYSROOT=iphoneos -DCMAKE_SYSTEM_NAME=iOS --build ..) so hopefully it's easy to compile for arm on an arm mac with a simple cmake invocation like cmake --build ...
benmkw
commented
3 years ago I'm getting the following:
λ (master|✔️ ) cmake .. -GNinja 5ms
-- The CXX compiler identification is AppleClang 12.0.0.12000032
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- CMAKE_BUILD_TYPE is not specified. Using default: RelWithDebInfo
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- TBBBind build target is disabled due to unsupported environment
-- The C compiler identification is AppleClang 12.0.0.12000032
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/benediktmandelkow/Downloads/oneTBB/build
i~/D/o/build
λ (master|✔️ ) ninja 2s 412ms
[1/304] Building CXX object src/tbb/CMakeFiles/tbb.dir/allocator.cpp.o
FAILED: src/tbb/CMakeFiles/tbb.dir/allocator.cpp.o
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -D__TBB_BUILD -I../src/tbb/../../include -O2 -g -DNDEBUG -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -fPIC -MMD -Wall -Wextra -Werror -Wno-parentheses -Wno-non-virtual-dtor -Wno-dangling-else -D_XOPEN_SOURCE -mrtm -std=c++11 -MD -MT src/tbb/CMakeFiles/tbb.dir/allocator.cpp.o -MF src/tbb/CMakeFiles/tbb.dir/allocator.cpp.o.d -o src/tbb/CMakeFiles/tbb.dir/allocator.cpp.o -c ../src/tbb/allocator.cpp
clang: error: argument unused during compilation: '-mrtm' [-Werror,-Wunused-command-line-argument]
clang: error: argument unused during compilation: '-MD' [-Werror,-Wunused-command-line-argument]
[2/304] Building CXX object src/tbb/CMakeFiles/tbb.dir/global_control.cpp.o
FAILED: src/tbb/CMakeFiles/tbb.dir/global_control.cpp.o with make
λ (master|✔️ ) make 4ms
Scanning dependencies of target tbb
[ 0%] Building CXX object src/tbb/CMakeFiles/tbb.dir/allocator.cpp.o
clang: error: argument unused during compilation: '-mrtm' [-Werror,-Wunused-command-line-argument]
make[2]: *** [src/tbb/CMakeFiles/tbb.dir/allocator.cpp.o] Error 1
make[1]: *** [src/tbb/CMakeFiles/tbb.dir/all] Error 2
make: *** [all] Error 2As a side note: Apparently gcc implements the parallel TS (clang does not) but gcc does not yet work on arm macs. Maybe once gcc works that might be worth looking into instead of pstl. Or maybe using clang with gcc libstdc++, not sure if this will work but...
michaeleisel
commented
3 years ago Try with -Wno-error, that looks like just a warning
benmkw
commented
3 years ago Tried but it ended up setting the flag to early and was thus unused so I did that:
diff --git a/cmake/compilers/AppleClang.cmake b/cmake/compilers/AppleClang.cmake
index af964e66..9d2edfc6 100644
--- a/cmake/compilers/AppleClang.cmake
+++ b/cmake/compilers/AppleClang.cmake
@@ -14,7 +14,6 @@
set(TBB_LINK_DEF_FILE_FLAG -Wl,-exported_symbols_list,)
set(TBB_DEF_FILE_PREFIX mac${TBB_ARCH})
-set(TBB_WARNING_LEVEL -Wall -Wextra $<$<BOOL:${TBB_STRICT}>:-Werror>)
set(TBB_TEST_WARNING_FLAGS -Wshadow -Wcast-qual -Woverloaded-virtual -Wnon-virtual-dtor)
set(TBB_WARNING_SUPPRESS -Wno-parentheses -Wno-non-virtual-dtor -Wno-dangling-else)
# For correct ucontext.h structures layoutand it worked
(without DTBB_TEST=OFF, many errors happen ;) )
thii
commented
3 years ago It looks like you built the latest version of tbb? zld is using the version of tbb at v2020.1.
thii
commented
3 years ago Submitted https://github.com/michaeleisel/zld/pull/75 that builds tbb from source.
darthdeus
commented
3 years ago I'm not sure if I'm doing something wrong, but it still doesn't build for me
==> Installing zld from michaeleisel/zld
==> Downloading https://github.com/michaeleisel/zld/archive/1.2.1.zip
Already downloaded: /Users/darth/Library/Caches/Homebrew/downloads/08102dd0a3edd52b3530145d276a75a94485f0fd9a1ee5828b18d262a5c9986f--zld-1.2.1.zip
==> make clean
==> make build
Last 15 lines from /Users/darth/Library/Logs/Homebrew/zld/02.make:
absl::lts_2020_02_25::container_internal::raw_hash_set<absl::lts_2020_02_25::container_internal::FlatHashMapPolicy<ld::Atom const*, unsigned long>, absl::lts_2020_02_25::container_internal::HashEq<ld::Atom const*, void>::Hash, absl::lts_2020_02_25::container_internal::HashEq<ld::Atom const*, void>::Eq, std::__1::allocator<std::__1::pair<ld::Atom const* const, unsigned long> > >::resize(unsigned long) in code_dedup.o
absl::lts_2020_02_25::container_internal::raw_hash_set<absl::lts_2020_02_25::container_internal::FlatHashMapPolicy<ld::Atom const*, unsigned long>, absl::lts_2020_02_25::container_internal::HashEq<ld::Atom const*, void>::Hash, absl::lts_2020_02_25::container_internal::HashEq<ld::Atom const*, void>::Eq, std::__1::allocator<std::__1::pair<ld::Atom const* const, unsigned long> > >::drop_deletes_without_resize() in code_dedup.o
std::__1::pair<unsigned long, bool> absl::lts_2020_02_25::container_internal::raw_hash_set<absl::lts_2020_02_25::container_internal::FlatHashMapPolicy<ld::Atom const*, ld::Atom const*>, absl::lts_2020_02_25::container_internal::HashEq<ld::Atom const*, void>::Hash, absl::lts_2020_02_25::container_internal::HashEq<ld::Atom const*, void>::Eq, std::__1::allocator<std::__1::pair<ld::Atom const* const, ld::Atom const*> > >::find_or_prepare_insert<ld::Atom const*>(ld::Atom const* const&) in code_dedup.o
absl::lts_2020_02_25::container_internal::raw_hash_set<absl::lts_2020_02_25::container_internal::FlatHashMapPolicy<ld::Atom const*, ld::Atom const*>, absl::lts_2020_02_25::container_internal::HashEq<ld::Atom const*, void>::Hash, absl::lts_2020_02_25::container_internal::HashEq<ld::Atom const*, void>::Eq, std::__1::allocator<std::__1::pair<ld::Atom const* const, ld::Atom const*> > >::resize(unsigned long) in code_dedup.o
...
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
** BUILD FAILED **
The following build commands failed:
Ld /tmp/zld-20201229-29199-120p591/zld-1.2.1/build/Build/Products/Release/zld normal
(1 failure)
make: *** [build] Error 65
It's not released yet.
benmkw
commented
3 years ago i was told that the arm linker signs while linking,
Currently building rust programs that use build.rs (so basically any program which has a few dependencies) does not work as well as things like cargo run
cargo run
Compiling helloworld v0.1.0 (/Users/benediktmandelkow/Downloads/helloworld)
Finished dev [unoptimized + debuginfo] target(s) in 0.46s
Running `target/debug/helloworld`
fish: 'cargo run' terminated by signal SIGKILL (Forced quit)
./target/debug/helloworld
fish: './target/debug/helloworld' terminated by signal SIGKILL (Forced quit)If someone wants to work on this, one implementation of how to compute the signature is here (MIT License) https://github.com/kubkon/zld/blob/main/src/CodeSignature.zig
I get the following when building on m1 arm mac:
It seems the issue is basically just that the target is set to x86 instead of arm so the fix could maybe be simple. Unfortunately I don't really know much about Xcode or how you set up the build yet. If you have a diff/ branch/ something I could/ should try I'm happy to do that and give feedback/ give more information.