Open troian opened 2 years ago
The problem is arm64e, it’s a very unstable API it should really be removed from osxcross as even on macOS Xcode uses arm64 not arm64e (only binaries from Apple even use arm64e)
hm, thats interesting, have you got any more details on that?
i'll remove arm64e
but still curious why it works when building amd64
docker image on amd64
host
hm, thats interesting, have you got any more details on that?
Not really, arm64e and x86_64h are not used on macOS by anything expect maybe Apple (not checked for an x86_64h slice on my M1 yet)
i'll remove
arm64e
but still curious why it works when buildingamd64
docker image onamd64
host
Within that docker image I’d assume it’s just falling back to the arm64 symbols same for x86_64h.
I had tried the initial test for this and the arm64e binary failed to run, can’t remember about the x86_64h ad that wasn’t what was asked.
Maybe Apple Clang/ld64 is attempting to be over smart and tripping itself up.
Outside of some weird configuration I’m not even sure why anyone would want to compile osxcross to target arm64 in any Mac that can run macOS Big Sur or later as that can all be done natively. Hell you can even still target i386 with a 10.13.SDK
it didn't help unfortunately. I think issue is rather with c++ symbols not installed. from logs clang++ fails on all of architectures.
#11 1835.2 testing x86_64-apple-darwin21.1-clang ... works
#11 1836.1 testing x86_64-apple-darwin21.1-clang++ ... Undefined symbols for architecture x86_64:
just to clarify. i'm trying to build amd64
image on arm64
host. building amd64
on amd64
host goes ok.
building arm64
image on both amd64
and arm64
hosts goes ok.
Can you do
file x86_64-apple-darwin21.1-clang++
And see what architecture it was compiled for, also do the same for arm64-apple-darwin21.1-clang++
root@35c0d2824791:/osxcross/target/bin# file x86_64-apple-darwin21.1-clang++
x86_64-apple-darwin21.1-clang++: symbolic link to x86_64-apple-darwin21.1-wrapper
root@35c0d2824791:/osxcross/target/bin# file x86_64-apple-darwin21.1-wrapper
x86_64-apple-darwin21.1-wrapper: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=eb0e65ab0b9f9b01c999a0268929bad4c0f4ccc1, for GNU/Linux 3.2.0, not stripped
root@35c0d2824791:/osxcross/target/bin# file arm64-apple-darwin21.1-clang++
arm64-apple-darwin21.1-clang++: symbolic link to x86_64-apple-darwin21.1-wrapper
root@35c0d2824791:/osxcross/target/bin# file x86_64-apple-darwin21.1-wrapper
x86_64-apple-darwin21.1-wrapper: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=eb0e65ab0b9f9b01c999a0268929bad4c0f4ccc1, for GNU/Linux 3.2.0, not stripped
just feels like c++ symbols are missing
How was the SDK package generated?, let’s try ruling that out.
downloaded 11.3 from here and 12.0 from here
wonder if absence of libstdc++ in sdk > 10.14 could be case
downloaded 11.3 from here and 12.0 from here
The SDKs from phracker are only useful for use on actual macOS using XCode. (Though 10.13 was broken last I checked)
The SDKs from joseluisq should be packaged correctly but I’ve not had chance to verify them yet.
wonder if absence of libstdc++ in sdk > 10.14 could be case
That’s another matter entirely and only really matter if your attempting to target basically ancient versions of Mac OSX, that was the default before Mac OSX 10.7 added libc++ that eventually become the default.
As you mention having access to actual Mac hardware you should instead generate the SDK yourself to ensure it was done correctly.
apparently something is missing where libraries search occurs
this works
x86_64-apple-darwin21.1-clang++ ./oclang/test.cpp -O2 -Wall -L/osxcross/target/SDK/MacOSX12.0.sdk/usr/lib -lc++ -o test
does not work. note there is no -lc++
(same as in)
x86_64-apple-darwin21.1-clang++ ./oclang/test.cpp -O2 -Wall -L/osxcross/target/SDK/MacOSX12.0.sdk/usr/lib -o test
Hum that does seem strange.
Try setting the following in the environment OSXCROSS_MP_INC="YES"
and not setting -L/osxcross/target/SDK/MacOSX12.0.sdk/usr/lib -lc++
That should inject everything but it’s possible that Apple changed things again meaning that passing -lc++
became required.
does not help, weirdly
There shouldn’t be a need to pass the SDK location, the -clang wrapper should at least inject that by default so maybe just pass -lc++
would be enough.
you're right, SDK location is unnecessary. ended up with small adjustment to build scripts
diff --git a/build.sh b/build.sh
index bbcff0c..d26b7c5 100755
--- a/build.sh
+++ b/build.sh
@@ -318,22 +318,22 @@ fi
if [ $X86_64H_SUPPORTED -eq 1 ]; then
test_compiler x86_64h-apple-$TARGET-clang $BASE_DIR/oclang/test.c
- test_compiler x86_64h-apple-$TARGET-clang++ $BASE_DIR/oclang/test.cpp
+ test_compiler_cxx x86_64h-apple-$TARGET-clang++ $BASE_DIR/oclang/test.cpp
echo ""
fi
if [ $ARM_SUPPORTED -eq 1 ]; then
test_compiler arm64-apple-$TARGET-clang $BASE_DIR/oclang/test.c
- test_compiler arm64-apple-$TARGET-clang++ $BASE_DIR/oclang/test.cpp
+ test_compiler_cxx arm64-apple-$TARGET-clang++ $BASE_DIR/oclang/test.cpp
echo ""
test_compiler arm64e-apple-$TARGET-clang $BASE_DIR/oclang/test.c
- test_compiler arm64e-apple-$TARGET-clang++ $BASE_DIR/oclang/test.cpp
+ test_compiler_cxx arm64e-apple-$TARGET-clang++ $BASE_DIR/oclang/test.cpp
echo ""
fi
test_compiler x86_64-apple-$TARGET-clang $BASE_DIR/oclang/test.c "required"
-test_compiler x86_64-apple-$TARGET-clang++ $BASE_DIR/oclang/test.cpp "required"
+test_compiler_cxx x86_64-apple-$TARGET-clang++ $BASE_DIR/oclang/test.cpp "required"
echo ""
echo "Do not forget to add"
diff --git a/tools/tools.sh b/tools/tools.sh
index a09ec64..55a1e81 100644
--- a/tools/tools.sh
+++ b/tools/tools.sh
@@ -461,6 +461,27 @@ function test_compiler()
fi
}
+function test_compiler_cxx()
+{
+ if [ "$3" != "required" ]; then
+ set +e
+ fi
+
+ echo -ne "testing $1 ... "
+ $1 $2 -O2 -Wall -lc++ -o test
+
+ if [ $? -eq 0 ]; then
+ rm test
+ echo "works"
+ else
+ echo "failed (ignored)"
+ fi
+
+ if [ "$3" != "required" ]; then
+ set -e
+ fi
+}
+
function test_compiler_cxx11()
{
set +e
I don’t believe editing the test to pass -lc++
is a good idea as a later compile using c++ will end up failing where the prior test failed.
I believe the more correct way would be having the clang wrapper passing -lc++
when -clang++ wrapper is used.
Hum if the wrapper ends with -libc++
then it should inject that by default.
From a quick look over it does seem that’s the case. I guess the next thing to figure out is what’s the first SDK that required passing -lc++
then it can be injected accordingly.
as far as i can tell 11.1
, 11.3
and 12.0
are having same issue when using docker build --platform=linux/amd64
run on arm64
. sorry i won't probably check other versions as it takes quite a bit of time.
I’ll attempt to do additional verifications when I’m free.
But if that already gets things working then awesome job.
i'm building multiarch golang-cross image. originally image was built to run on
amd64
but as M1 macs are broadbanding now i wanna create multiarch image.amd64
on Intel machine works fineaarch64
on M1 works fineamd64
on M1 fails with linker error below.any thoughts what i am missing?
linker errors
``` #11 1735.1 mv wrapper /osxcross/target/bin/x86_64-apple-darwin20.4-wrapper #11 1735.2 create_symlink x86_64-apple-darwin20.4-wrapper x86_64-apple-darwin20.4-clang #11 1735.2 create_symlink x86_64-apple-darwin20.4-wrapper x86_64h-apple-darwin20.4-clang #11 1735.2 create_symlink x86_64-apple-darwin20.4-wrapper aarch64-apple-darwin20.4-clang #11 1735.2 create_symlink x86_64-apple-darwin20.4-wrapper arm64-apple-darwin20.4-clang #11 1735.2 create_symlink x86_64-apple-darwin20.4-wrapper arm64e-apple-darwin20.4-clang #11 1735.2 create_symlink x86_64-apple-darwin20.4-wrapper o64-clang #11 1735.3 create_symlink x86_64-apple-darwin20.4-wrapper o64h-clang #11 1735.3 create_symlink x86_64-apple-darwin20.4-wrapper oa64-clang #11 1735.3 create_symlink x86_64-apple-darwin20.4-wrapper oa64e-clang #11 1735.3 create_symlink x86_64-apple-darwin20.4-wrapper x86_64-apple-darwin20.4-clang++ #11 1735.3 create_symlink x86_64-apple-darwin20.4-wrapper x86_64h-apple-darwin20.4-clang++ #11 1735.3 create_symlink x86_64-apple-darwin20.4-wrapper aarch64-apple-darwin20.4-clang++ #11 1735.3 create_symlink x86_64-apple-darwin20.4-wrapper arm64-apple-darwin20.4-clang++ #11 1735.4 create_symlink x86_64-apple-darwin20.4-wrapper arm64e-apple-darwin20.4-clang++ #11 1735.4 create_symlink x86_64-apple-darwin20.4-wrapper o64-clang++ #11 1735.4 create_symlink x86_64-apple-darwin20.4-wrapper o64h-clang++ #11 1735.4 create_symlink x86_64-apple-darwin20.4-wrapper oa64-clang++ #11 1735.4 create_symlink x86_64-apple-darwin20.4-wrapper oa64e-clang++ #11 1735.4 create_symlink x86_64-apple-darwin20.4-wrapper x86_64-apple-darwin20.4-clang++-libc++ #11 1735.4 create_symlink x86_64-apple-darwin20.4-wrapper x86_64h-apple-darwin20.4-clang++-libc++ #11 1735.5 create_symlink x86_64-apple-darwin20.4-wrapper aarch64-apple-darwin20.4-clang++-libc++ #11 1735.5 create_symlink x86_64-apple-darwin20.4-wrapper arm64-apple-darwin20.4-clang++-libc++ #11 1735.5 create_symlink x86_64-apple-darwin20.4-wrapper arm64e-apple-darwin20.4-clang++-libc++ #11 1735.5 create_symlink x86_64-apple-darwin20.4-wrapper o64-clang++-libc++ #11 1735.5 create_symlink x86_64-apple-darwin20.4-wrapper o64h-clang++-libc++ #11 1735.5 create_symlink x86_64-apple-darwin20.4-wrapper oa64-clang++-libc++ #11 1735.5 create_symlink x86_64-apple-darwin20.4-wrapper oa64e-clang++-libc++ #11 1735.6 create_symlink x86_64-apple-darwin20.4-wrapper x86_64-apple-darwin20.4-clang++-stdc++ #11 1735.6 create_symlink x86_64-apple-darwin20.4-wrapper x86_64h-apple-darwin20.4-clang++-stdc++ #11 1735.6 create_symlink x86_64-apple-darwin20.4-wrapper aarch64-apple-darwin20.4-clang++-stdc++ #11 1735.6 create_symlink x86_64-apple-darwin20.4-wrapper arm64-apple-darwin20.4-clang++-stdc++ #11 1735.6 create_symlink x86_64-apple-darwin20.4-wrapper arm64e-apple-darwin20.4-clang++-stdc++ #11 1735.6 create_symlink x86_64-apple-darwin20.4-wrapper o64-clang++-stdc++ #11 1735.7 create_symlink x86_64-apple-darwin20.4-wrapper o64h-clang++-stdc++ #11 1735.7 create_symlink x86_64-apple-darwin20.4-wrapper oa64-clang++-stdc++ #11 1735.7 create_symlink x86_64-apple-darwin20.4-wrapper oa64e-clang++-stdc++ #11 1735.7 create_symlink x86_64-apple-darwin20.4-wrapper x86_64-apple-darwin20.4-clang++-gstdc++ #11 1735.7 create_symlink x86_64-apple-darwin20.4-wrapper o64-clang++-gstdc++ #11 1735.7 create_symlink x86_64-apple-darwin20.4-wrapper oa64-clang++-gstdc++ #11 1735.7 create_symlink x86_64-apple-darwin20.4-wrapper oa64e-clang++-gstdc++ #11 1735.8 create_symlink x86_64-apple-darwin20.4-wrapper x86_64-apple-darwin20.4-cc #11 1735.8 create_symlink x86_64-apple-darwin20.4-wrapper x86_64h-apple-darwin20.4-cc #11 1735.8 create_symlink x86_64-apple-darwin20.4-wrapper aarch64-apple-darwin20.4-cc #11 1735.8 create_symlink x86_64-apple-darwin20.4-wrapper arm64-apple-darwin20.4-cc #11 1735.8 create_symlink x86_64-apple-darwin20.4-wrapper arm64e-apple-darwin20.4-cc #11 1735.8 create_symlink x86_64-apple-darwin20.4-wrapper x86_64-apple-darwin20.4-c++ #11 1735.9 create_symlink x86_64-apple-darwin20.4-wrapper x86_64h-apple-darwin20.4-c++ #11 1735.9 create_symlink x86_64-apple-darwin20.4-wrapper aarch64-apple-darwin20.4-c++ #11 1735.9 create_symlink x86_64-apple-darwin20.4-wrapper arm64-apple-darwin20.4-c++ #11 1735.9 create_symlink x86_64-apple-darwin20.4-wrapper arm64e-apple-darwin20.4-c++ #11 1735.9 create_symlink x86_64-apple-darwin20.4-wrapper osxcross #11 1735.9 create_symlink x86_64-apple-darwin20.4-wrapper x86_64-apple-darwin20.4-osxcross #11 1735.9 create_symlink x86_64-apple-darwin20.4-wrapper x86_64h-apple-darwin20.4-osxcross #11 1736.0 create_symlink x86_64-apple-darwin20.4-wrapper aarch64-apple-darwin20.4-osxcross #11 1736.0 create_symlink x86_64-apple-darwin20.4-wrapper arm64-apple-darwin20.4-osxcross #11 1736.0 create_symlink x86_64-apple-darwin20.4-wrapper arm64e-apple-darwin20.4-osxcross #11 1736.0 create_symlink x86_64-apple-darwin20.4-wrapper osxcross-conf #11 1736.0 create_symlink x86_64-apple-darwin20.4-wrapper x86_64-apple-darwin20.4-osxcross-conf #11 1736.0 create_symlink x86_64-apple-darwin20.4-wrapper x86_64h-apple-darwin20.4-osxcross-conf #11 1736.0 create_symlink x86_64-apple-darwin20.4-wrapper aarch64-apple-darwin20.4-osxcross-conf #11 1736.1 create_symlink x86_64-apple-darwin20.4-wrapper arm64-apple-darwin20.4-osxcross-conf #11 1736.1 create_symlink x86_64-apple-darwin20.4-wrapper arm64e-apple-darwin20.4-osxcross-conf #11 1736.1 create_symlink x86_64-apple-darwin20.4-wrapper osxcross-env #11 1736.1 create_symlink x86_64-apple-darwin20.4-wrapper x86_64-apple-darwin20.4-osxcross-env #11 1736.1 create_symlink x86_64-apple-darwin20.4-wrapper x86_64h-apple-darwin20.4-osxcross-env #11 1736.1 create_symlink x86_64-apple-darwin20.4-wrapper aarch64-apple-darwin20.4-osxcross-env #11 1736.1 create_symlink x86_64-apple-darwin20.4-wrapper arm64-apple-darwin20.4-osxcross-env #11 1736.2 create_symlink x86_64-apple-darwin20.4-wrapper arm64e-apple-darwin20.4-osxcross-env #11 1736.2 create_symlink x86_64-apple-darwin20.4-wrapper osxcross-cmp #11 1736.2 create_symlink x86_64-apple-darwin20.4-wrapper x86_64-apple-darwin20.4-osxcross-cmp #11 1736.2 create_symlink x86_64-apple-darwin20.4-wrapper x86_64h-apple-darwin20.4-osxcross-cmp #11 1736.2 create_symlink x86_64-apple-darwin20.4-wrapper aarch64-apple-darwin20.4-osxcross-cmp #11 1736.2 create_symlink x86_64-apple-darwin20.4-wrapper arm64-apple-darwin20.4-osxcross-cmp #11 1736.3 create_symlink x86_64-apple-darwin20.4-wrapper arm64e-apple-darwin20.4-osxcross-cmp #11 1736.3 create_symlink x86_64-apple-darwin20.4-wrapper osxcross-man #11 1736.3 create_symlink x86_64-apple-darwin20.4-wrapper x86_64-apple-darwin20.4-osxcross-man #11 1736.3 create_symlink x86_64-apple-darwin20.4-wrapper x86_64h-apple-darwin20.4-osxcross-man #11 1736.3 create_symlink x86_64-apple-darwin20.4-wrapper aarch64-apple-darwin20.4-osxcross-man #11 1736.3 create_symlink x86_64-apple-darwin20.4-wrapper arm64-apple-darwin20.4-osxcross-man #11 1736.3 create_symlink x86_64-apple-darwin20.4-wrapper arm64e-apple-darwin20.4-osxcross-man #11 1736.4 create_symlink x86_64-apple-darwin20.4-wrapper x86_64-apple-darwin20.4-pkg-config #11 1736.4 create_symlink x86_64-apple-darwin20.4-wrapper x86_64h-apple-darwin20.4-pkg-config #11 1736.4 create_symlink x86_64-apple-darwin20.4-wrapper aarch64-apple-darwin20.4-pkg-config #11 1736.4 create_symlink x86_64-apple-darwin20.4-wrapper arm64-apple-darwin20.4-pkg-config #11 1736.4 create_symlink x86_64-apple-darwin20.4-wrapper arm64e-apple-darwin20.4-pkg-config #11 1736.4 create_symlink x86_64-apple-darwin20.4-wrapper sw_vers #11 1736.4 create_symlink x86_64-apple-darwin20.4-wrapper x86_64-apple-darwin20.4-sw_vers #11 1736.5 create_symlink x86_64-apple-darwin20.4-wrapper x86_64h-apple-darwin20.4-sw_vers #11 1736.5 create_symlink x86_64-apple-darwin20.4-wrapper aarch64-apple-darwin20.4-sw_vers #11 1736.5 create_symlink x86_64-apple-darwin20.4-wrapper arm64-apple-darwin20.4-sw_vers #11 1736.5 create_symlink x86_64-apple-darwin20.4-wrapper arm64e-apple-darwin20.4-sw_vers #11 1736.6 create_symlink /usr/bin/dsymutil x86_64-apple-darwin20.4-dsymutil #11 1736.6 create_symlink /usr/bin/dsymutil x86_64h-apple-darwin20.4-dsymutil #11 1736.6 create_symlink x86_64-apple-darwin20.4-wrapper xcrun #11 1736.6 create_symlink x86_64-apple-darwin20.4-wrapper x86_64-apple-darwin20.4-xcrun #11 1736.6 create_symlink x86_64-apple-darwin20.4-wrapper x86_64h-apple-darwin20.4-xcrun #11 1736.6 create_symlink x86_64-apple-darwin20.4-wrapper aarch64-apple-darwin20.4-xcrun #11 1736.7 create_symlink x86_64-apple-darwin20.4-wrapper arm64-apple-darwin20.4-xcrun #11 1736.7 create_symlink x86_64-apple-darwin20.4-wrapper arm64e-apple-darwin20.4-xcrun #11 1736.7 create_symlink x86_64-apple-darwin20.4-wrapper xcodebuild #11 1736.7 create_symlink x86_64-apple-darwin20.4-wrapper x86_64-apple-darwin20.4-xcodebuild #11 1736.7 create_symlink x86_64-apple-darwin20.4-wrapper x86_64h-apple-darwin20.4-xcodebuild #11 1736.7 create_symlink x86_64-apple-darwin20.4-wrapper aarch64-apple-darwin20.4-xcodebuild #11 1736.7 create_symlink x86_64-apple-darwin20.4-wrapper arm64-apple-darwin20.4-xcodebuild #11 1736.7 create_symlink x86_64-apple-darwin20.4-wrapper arm64e-apple-darwin20.4-xcodebuild #11 1736.8 #11 1737.0 #11 1737.0 testing x86_64-apple-darwin20.4-clang++ -stdlib=libc++ -std=c++11 ... failed (ignored) #11 1741.2 #11 1741.2 testing x86_64h-apple-darwin20.4-clang ... works #11 1742.0 testing x86_64h-apple-darwin20.4-clang++ ... Undefined symbols for architecture x86_64h: #11 1745.8 "std::__1::locale::use_facet(std::__1::locale::id&) const", referenced from: #11 1745.8 _main in test-981283.o #11 1745.8 std::__1::basic_ostreamdockerfile stages
```Dockerfile FROM base AS osx-cross-base ARG DEBIAN_FRONTEND=noninteractive # Install deps RUN \ set -x; \ echo "Starting image build for Debian Bullseye" \ && dpkg --add-architecture amd64 \ && dpkg --add-architecture arm64 \ && dpkg --add-architecture armel \ && dpkg --add-architecture armhf \ && dpkg --add-architecture i386 \ && dpkg --add-architecture mips \ && dpkg --add-architecture mipsel \ && dpkg --add-architecture powerpc \ && dpkg --add-architecture ppc64el \ && apt-get update \ && apt-get install --no-install-recommends -y -q \ autoconf \ automake \ bc \ binfmt-support \ binutils-multiarch \ build-essential \ clang \ gcc \ g++ \ gdb \ mingw-w64 \ crossbuild-essential-amd64 \ crossbuild-essential-arm64 \ crossbuild-essential-armel \ crossbuild-essential-armhf \ crossbuild-essential-mipsel \ crossbuild-essential-ppc64el \ devscripts \ libtool \ llvm \ multistrap \ patch \ qemu-user-static \ xz-utils \ mercurial \ && apt -y autoremove \ && apt-get clean \ && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* FROM osx-cross-base AS osx-cross ARG OSX_CROSS_COMMIT WORKDIR "${OSX_CROSS_PATH}" # install osxcross: RUN \ git clone https://github.com/tpoechtrager/osxcross.git . \ && git checkout -q "${OSX_CROSS_COMMIT}" \ && rm -rf ./.git COPY --from=osx-sdk "${OSX_CROSS_PATH}/." "${OSX_CROSS_PATH}/" ARG OSX_VERSION_MIN RUN \ apt-get update \ && apt-get install --no-install-recommends -y -q \ autotools-dev \ binutils-multiarch-dev \ libxml2-dev \ lzma-dev \ libssl-dev \ zlib1g-dev \ libmpc-dev \ libmpfr-dev \ libgmp-dev \ musl-tools \ && apt -y autoremove \ && apt-get clean \ && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* \ && UNATTENDED=yes OSX_VERSION_MIN=${OSX_VERSION_MIN} ./build.sh ```