Open niklassaers opened 5 years ago
niklassaers
commented
5 years ago (of course, I could be heading down the entirely wrong path here. If I only compile OpenSSL using marcelosalloum/OpenSSL-for-iPhone with targets "ios-sim-cross-i386 ios64-cross-arm64 ios-cross-armv7s ios-cross-armv7 mac-catalyst-x86_64", they import just fine. But, of course, I need libssh2 too)
Jan-E
commented
5 years ago It may be my lack of knowledge of Catalyst, but is not the only architecture you need for Catalyst on OSX 10.15 x86_64?
tfr@TFRs-Mac iSSH2 % ./iSSH2.sh --platform=iphoneos --min-version=8.0 --archs="x86_64"
Initializing...
Libssh2 version: 1.9.0 (Automatically detected)
OpenSSL version: 1.1.1d (Automatically detected)
SDK version: 13.2 (Automatically detected)
Architectures: x86_64
Platform: iphoneos
Platform min version: 8.0
Downloading openssl-1.1.1d.tar.gz
############################################################################################# 100.0%
MD5: 3be209000dbc7e1b95bcdf47980a3baa
Extracting openssl-1.1.1d.tar.gz
Building OpenSSL 1.1.1d, please wait...
- iPhoneSimulator x86_64 done!
Building libssl.a fat library...
Building libcrypto.a fat library...
Copying headers...
Building done.
Downloading libssh2-1.9.0.tar.gz
############################################################################################# 100.0%############################################################################################# 100.0%
MD5: 1beefafe8963982adc84b408b2959927
Extracting libssh2-1.9.0.tar.gz
Building Libssh2 1.9.0:
- iPhoneSimulator x86_64 done!
Building libssh2.a fat library...
Copying headers...
Building done.
Cleaning up temporary files...
tfr@TFRs-Mac iSSH2 % xcodebuild -create-xcframework \
-library libssh2_macosx/lib/libssh2.a -headers libssh2_macosx/include \
-library libssh2_iphoneos/lib/libssh2.a -headers libssh2_iphoneos/include \
-output libssh2.xcframework
xcframework successfully written out to: /Users/tfr/Documents/iSSH2/libssh2.xcframeworkOr could you just use the libssh2 for macOS for a Catalyst app?
niklassaers
commented
5 years ago Hi @Jan-E , Thanks for your input. Making a framework with only the simulator part (--platform=iphoneos --archs="x86_64") and macOS part makes the iOS binary that is dependent on arm64 not able to compile. In Xcode it shows me the following error: "While building for Mac Catalyst, no library for this platform was found in 'libssh2.xcframework'."
I've tried embedding separate libssh2.a files for arm64 and for macOS, but I keep getting "Building for Mac Catalyst, but the linked library 'libssh2.a' was built for iOS + iOS Simulator. You may need to restrict the platforms for which this library should be linked in the target editor, or replace it with an XCFramework that supports both platforms."
Lipo'ing them together into one gave the added trouble that both the simulator and macOS versions were x86_64, and removing the simulator part got me back to the error above. So apparently x86_64 in the simulator and x86_64 on macOS is not the same in its eyes. That's why I brought up the targets from the OpenSSL-for-iPhone project above: "ios-sim-cross-i386 ios64-cross-arm64 ios-cross-armv7s ios-cross-armv7 mac-catalyst-x86_64". Compiling it gives me static libraries, no .xcframework, and those static libraries just work. :-) So if we could only bring this to libssh2.a too, that should solve the problem too.
For the xcframework route - I tried adding the following to its Info.plist and copying the files into place myself, but alas I'm back at "While building for Mac Catalyst, no library for this platform was found in 'libssh2.xcframework'." The lines I added were
<dict>
<key>HeadersPath</key>
<string>Headers</string>
<key>LibraryIdentifier</key>
<string>ios-arm64</string>
<key>LibraryPath</key>
<string>libssh2.a</string>
<key>SupportedArchitectures</key>
<array>
<string>arm64</string>
<string>arm64e</string>
</array>
<key>SupportedPlatform</key>
<string>ios</string>
</dict>OK, clear. When I have got the time I'll experiment further, but for the moment I am out of ideas. Wishing you the best with your attempts.
niklassaers
commented
5 years ago I have made a demo catalyst app with OpenSSL integrated fine and the libssh2.a compiled as described above: https://github.com/niklassaers/DemoCatalyst
Jan-E
commented
5 years ago Getting closer. See https://github.com/Jan-E/iSSH2/commit/4262507e4512344b7173950179301c15934139bd and https://github.com/Jan-E/iSSH2/commit/4262507e4512344b7173950179301c15934139bd#commitcomment-35983917 for a working example of building a MacCatalyst library of libssh2.
The trick of building for MacCatalyst is adding -target x86_64-apple-ios13.0-macabi to the CFLAGS. See the explanation in https://stackoverflow.com/a/56860199/872051 where I got the idea to solve this.
Jan-E
commented
5 years ago I tried to use these libs in a real iPad app:
Ignoring file Pods/NMSSH/NMSSH-iOS/Libraries/lib/libssh2.a, missing required architecture armv7 in file Pods/NMSSH/NMSSH-iOS/Libraries/lib/libssh2.a (1 slices)Ergo, the other libs have to be added to the fat library as well.
Jan-E
commented
5 years ago https://github.com/Jan-E/iSSH2/commit/5f233a1a2d0276c1f28aef75647b651ec51a905a brought me closer to compiling arm64 for Mac Catalyst.
Build command:
CFLAGS="-target arm64-apple-ios13.0-macabi" ./iSSH2.sh --platform=iphoneos --target=iphoneos --min-version=10.15 --archs="arm64"The trick is to add -target arm64-apple-ios13.0-macabi to the CFLAGS in the Makefile's after the configure. This went fine up intil the linking of libSystem.tbd
libtool: link: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -arch arm64 -pipe -no-cpp-precomp -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.2.sdk -target arm64-apple-ios13.0-macabi -miphoneos-version-min=10.15 -fembed-bitcode -DLIBSSH2_DARWIN -o ssh2 ssh2.o ../src/.libs/libssh2.a -L/Users/tfr/documents/iSSH2-Catalyst/openssl_iphoneos//lib -lssl -lcrypto
clang: warning: overriding '-miphoneos-version-min=10.15' option with '--target=arm64-apple-ios13.0-macabi' [-Woverriding-t-option]
ld: warning: ignoring file /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.0/lib/darwin/libclang_rt.osx.a, missing required architecture arm64 in file /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.0/lib/darwin/libclang_rt.osx.a (3 slices)
ld: building for Mac Catalyst, but linking in .tbd file (/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.2.sdk/usr/lib/libSystem.tbd) built for iOS, file '/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.2.sdk/usr/lib/libSystem.tbd' for architecture arm64
Where to find a libSystem.tbd, built for Mac Catalyst?? There are about 8 variants of libSystem.tbd in the Xcode.app. I tried the most likely ones, but none of hem worked.
The missing required architecture arm64 in libclang_rt.osx.a is worrying as well, but it is only a warning, not a fatal error.
niklassaers
commented
5 years ago Thanks a lot, @Jan-E , great find with the CFLAGS and thanks for explaining how to use them. I'll follow your lead here and see what I can find out
Jan-E
commented
5 years ago I think I have solved it with this commit: https://github.com/Jan-E/iSSH2/commit/b9610a130e0ec71a395befd00e9810ac42d21cc1
There is a script catalyst.sh in the repo now that builds OpenSSL 1.1.1d and LibSSH2 1.9.0 for Mac Catalyst. The idea was stolen from the OpenSSL for iPhone repo: build x86_64 for Mac Catalyst and all the others just for iOS and create a fat library with those libs. You cannot combine x86_64 for iOS and x86_64 for Mac Catalyst in the same fat library, so I went for the Mac Catalyst version.
% lipo -info ./openssl_iphoneos/lib/libssl.a
Architectures in the fat file: ./openssl_iphoneos/lib/libssl.a are: armv7 armv7s i386 x86_64 arm64 arm64e where x86_64 is the MacOSX version. I added those lib and the headers to https://github.com/Jan-E/DemoCatalyst/commits/master
Jan-E
commented
5 years ago FWIW: I tested the libs in a B2B iPad app: it uploaded 15.0 MB in 3.99 seconds. The iPad app uses the camera, so I will have to update a lot more to make it work on Mac Catalyst.
niklassaers
commented
5 years ago Wonderful, checking it out and will report back! :-)
Jan-E
commented
5 years ago The iOS simulator does not work:
ld: warning: building for iOS Simulator, but linking in object file (Pods/NMSSH/NMSSH-iOS/Libraries/lib/libcrypto.a(aes_cbc.o)) built for Mac Catalyst
In my main app that is no real problem, because I cannot test the camera either.
niklassaers
commented
4 years ago Wonderful work. My initial report is that it's working well! :-) I'll look into getting the simulator back up and running. In the catalyst world we could probably reduce build time a bit by skipping i386?
Jan-E
commented
4 years ago FWIW: Yesterday I built LibSSH2 1.9.0 & OpenSSL 1.1.1e for Mac Catalyst. I updated my Catalyst branch a little bit: https://github.com/Jan-E/iSSH2/commit/d157a25775b7c86354278843ba49e02bc4823105#commitcomment-37936052
@niklassaers Did you ever succeed in combining the libraries for Mac Catalyst and for iOS simulator?
niklassaers
commented
4 years ago Thanks. I didn't manage that, but I made one target that contained the catalyst binary and one target that contained the simulator binary, and then worked around it that way - two binaries, each with their own target
Jan-E
commented
3 years ago See https://github.com/Jan-E/iSSH2/commit/a76c6571c349105ddc4dcda0416c8e517c670806 for a mac Catalyst build using the Mac OSX SDK 11.0.
Jan-E
commented
3 years ago https://github.com/Jan-E/iSSH2/commit/ddb939119dd62d8284ad9333d3a1042d5f1d0879 works fine for OpenSSL 1.1.1k and LibSSH2 1.9.0 with Xcode 12.4
Jan-E
commented
3 years ago Jan-E
commented
3 years ago Xcode 12.5 implied an update of the macosx SDK from 11.1 to 11.3. That is automatically detected now by https://github.com/Jan-E/iSSH2/commit/52eb3f0f83e476d2f008f9f4881ccb2cbd8f0037
If you want to build Catalyst libs:
git clone -b catalyst https://github.com/Jan-E/iSSH2 iSSH2-catalyst
cd iSSH2-catalyst
./catalyst.shGrab a coffee or 2 and wait until the libs have been built with OpenSSL 1.1.1k and LibSSH2 1.9.0 (autodetected versions at this time).
Jan-E
commented
3 years ago Compiling with OpenSSL 1.1.1l went smooth. The end of running ./catalyst.sh:
Building Mac Catalyst libssh2 lib done.
Building fat files
lipo -create ${TMPDIR}iSSH2/libssh2-1.9.0/MacOSX_10.15-x86_64/install/lib/libssh2.a \
${TMPDIR}iSSH2/libssh2-1.9.0/iPhoneOS_14.5-arm64/install/lib/libssh2.a \
${TMPDIR}iSSH2/libssh2-1.9.0/iPhoneOS_14.5-arm64e/install/lib/libssh2.a \
${TMPDIR}iSSH2/libssh2-1.9.0/iPhoneOS_14.5-armv7/install/lib/libssh2.a \
${TMPDIR}iSSH2/libssh2-1.9.0/iPhoneOS_14.5-armv7s/install/lib/libssh2.a \
${TMPDIR}iSSH2/libssh2-1.9.0/iPhoneSimulator_14.5-i386/install/lib/libssh2.a -output ./libssh2_iphoneos/lib/libssh2.a
Architectures in the fat file: ./libssh2_iphoneos/lib/libssh2.a are: armv7 armv7s i386 x86_64 arm64 arm64e
lipo -create ${TMPDIR}iSSH2/openssl-1.1.1l/MacOSX_10.15-x86_64/install/lib/libcrypto.a \
${TMPDIR}iSSH2/openssl-1.1.1l/iPhoneOS_14.5-arm64/libcrypto.a \
${TMPDIR}iSSH2/openssl-1.1.1l/iPhoneOS_14.5-arm64e/libcrypto.a \
${TMPDIR}iSSH2/openssl-1.1.1l/iPhoneOS_14.5-armv7/libcrypto.a \
${TMPDIR}iSSH2/openssl-1.1.1l/iPhoneOS_14.5-armv7s/libcrypto.a \
${TMPDIR}iSSH2/openssl-1.1.1l/iPhoneSimulator_14.5-i386/libcrypto.a -output ./openssl_iphoneos/lib/libcrypto.a
Architectures in the fat file: ./openssl_iphoneos/lib/libcrypto.a are: armv7 armv7s i386 x86_64 arm64 arm64e
lipo -create ${TMPDIR}iSSH2/openssl-1.1.1l/MacOSX_10.15-x86_64/install/lib/libssl.a \
${TMPDIR}iSSH2/openssl-1.1.1l/iPhoneOS_14.5-arm64/libssl.a \
${TMPDIR}iSSH2/openssl-1.1.1l/iPhoneOS_14.5-arm64e/libssl.a \
${TMPDIR}iSSH2/openssl-1.1.1l/iPhoneOS_14.5-armv7/libssl.a \
${TMPDIR}iSSH2/openssl-1.1.1l/iPhoneOS_14.5-armv7s/libssl.a \
${TMPDIR}iSSH2/openssl-1.1.1l/iPhoneSimulator_14.5-i386/libssl.a -output ./openssl_iphoneos/lib/libssl.a
Architectures in the fat file: ./openssl_iphoneos/lib/libssl.a are: armv7 armv7s i386 x86_64 arm64 arm64e I had to update the script to auto-detect libSSH2 1.10.0, but then it went smooth:
Building Mac Catalyst openssl lib done. libssh2-1.10.0.tar.gz already exists MD5: f604ba083fad23bf715a9ecccc9f57f4 Extracting libssh2-1.10.0.tar.gz Building Libssh2 1.10.0: CRYPTO_BACKEND_OPTION = --with-crypto=openssl OPENSSLDIR = /Users/tfr/Documents/iSSH2-catalyst/openssl_iphoneos/ CFLAGS=-target x86_64-apple-ios13.0-macabi ./configure --host=x86_64-apple-darwin --prefix=/var/folders/l3/2krt_kt91sb5wkfdx52y3wtc0000gn/T/iSSH2/libssh2-1.10.0/MacOSX_10.15-x86_64/install --disable-debug --disable-dependency-tracking --disable-silent-rules --disable-examples-build --without-libz --with-crypto=openssl --with-libssl-prefix=/var/folders/l3/2krt_kt91sb5wkfdx52y3wtc0000gn/T//iSSH2/openssl-1.1.1l/MacOSX_10.15-x86_64/install --disable-shared --enable-static
Jan-E
commented
2 years ago Building Catalyst libs autodetected macOS SDK 12.1 & iOS SDK 15.2 (Xcode 13.2 on Monterey 12.1), OpenSSL 1.1.1m and LibSSH2 1.10.0. Everything OK in my fork:
git clone -b catalyst https://github.com/Jan-E/iSSH2 iSSH2-catalyst
cd iSSH2-catalyst
./catalyst.shGrab a coffee or 2 and wait until the libs have been built with OpenSSL and LibSSH2 for Mac Catalyst.
Jan-E
commented
2 years ago The macabi target had to be changed for Xcode 13.3. See https://github.com/Jan-E/iSSH2/commit/ec0bb5dea0d467acc57c5e23c52aec854d98ab89#commitcomment-68465816 for a complete session on building the Catalyst fat library
Jan-E
commented
2 years ago Update to OpenSSL 1.1.1n went without changing the scripts
Jan-E
commented
2 years ago Update to OpenSSL 1.1.1o went without changing the scripts
Jan-E
commented
2 years ago For the last ever libs that will support iOS 9.3.6 I dropped iPhoneSimulator i386. Building the catalyst branch with OpenSSL 1.1.1s ended as follows;
Building OpenSSL 1.1.1s x86_64 for darwin64-x86_64-cc, please wait...
SDK_VERSION = 12.3
- Mac Catalyst for darwin64-x86_64-cc x86_64 done!
Building libssl.a fat library...
Building libcrypto.a fat library...
Copying headers...
Building Mac Catalyst openssl lib done.
libssh2-1.10.0.tar.gz already exists
MD5: f604ba083fad23bf715a9ecccc9f57f4
Extracting libssh2-1.10.0.tar.gz
Building Libssh2 1.10.0:
CRYPTO_BACKEND_OPTION = --with-crypto=openssl
OPENSSLDIR = /Users/tfr/Documents/iSSH2-catalyst/openssl_iphoneos/
CFLAGS=-target x86_64-apple-ios-macabi ./configure --host=x86_64-apple-darwin --prefix=/var/folders/gk/blcd7mb502s_jshrvh6n_tnw0000gn/T/iSSH2/libssh2-1.10.0/MacOSX_10.15-x86_64/install --disable-debug --disable-dependency-tracking --disable-silent-rules --disable-examples-build --without-libz --with-crypto=openssl --with-libssl-prefix=/var/folders/gk/blcd7mb502s_jshrvh6n_tnw0000gn/T//iSSH2/openssl-1.1.1s/MacOSX_10.15-x86_64/install --disable-shared --enable-static
- MacOSX x86_64 done!
/var/folders/gk/blcd7mb502s_jshrvh6n_tnw0000gn/T/iSSH2/libssh2-1.10.0/MacOSX_10.15-x86_64/install/lib/libssh2.a
todo: lipo -create /var/folders/gk/blcd7mb502s_jshrvh6n_tnw0000gn/T/iSSH2/libssh2-1.10.0/MacOSX_10.15-x86_64/install/lib/libssh2.a -output /Users/tfr/Documents/iSSH2-catalyst/libssh2_iphoneos/lib/libssh2.a
Copying headers...
Building Mac Catalyst libssh2 lib done.
Building fat files
lipo -create ${TMPDIR}iSSH2/libssh2-1.10.0/MacOSX_10.15-x86_64/install/lib/libssh2.a \
${TMPDIR}iSSH2/libssh2-1.10.0/iPhoneOS_15.5-arm64/install/lib/libssh2.a \
${TMPDIR}iSSH2/libssh2-1.10.0/iPhoneOS_15.5-arm64e/install/lib/libssh2.a \
${TMPDIR}iSSH2/libssh2-1.10.0/iPhoneOS_15.5-armv7/install/lib/libssh2.a \
${TMPDIR}iSSH2/libssh2-1.10.0/iPhoneOS_15.5-armv7s/install/lib/libssh2.a -output ./libssh2_iphoneos/lib/libssh2.a
Architectures in the fat file: ./libssh2_iphoneos/lib/libssh2.a are: armv7 armv7s x86_64 arm64 arm64e
lipo -create ${TMPDIR}iSSH2/openssl-1.1.1s/MacOSX_10.15-x86_64/install/lib/libcrypto.a \
${TMPDIR}iSSH2/openssl-1.1.1s/iPhoneOS_15.5-arm64/libcrypto.a \
${TMPDIR}iSSH2/openssl-1.1.1s/iPhoneOS_15.5-arm64e/libcrypto.a \
${TMPDIR}iSSH2/openssl-1.1.1s/iPhoneOS_15.5-armv7/libcrypto.a \
${TMPDIR}iSSH2/openssl-1.1.1s/iPhoneOS_15.5-armv7s/libcrypto.a -output ./openssl_iphoneos/lib/libcrypto.a
Architectures in the fat file: ./openssl_iphoneos/lib/libcrypto.a are: armv7 armv7s x86_64 arm64 arm64e
lipo -create ${TMPDIR}iSSH2/openssl-1.1.1s/MacOSX_10.15-x86_64/install/lib/libssl.a \
${TMPDIR}iSSH2/openssl-1.1.1s/iPhoneOS_15.5-arm64/libssl.a \
${TMPDIR}iSSH2/openssl-1.1.1s/iPhoneOS_15.5-arm64e/libssl.a \
${TMPDIR}iSSH2/openssl-1.1.1s/iPhoneOS_15.5-armv7/libssl.a \
${TMPDIR}iSSH2/openssl-1.1.1s/iPhoneOS_15.5-armv7s/libssl.a -output ./openssl_iphoneos/lib/libssl.a
Architectures in the fat file: ./openssl_iphoneos/lib/libssl.a are: armv7 armv7s x86_64 arm64 arm64e
I updated https://github.com/Jan-E/iSSH2/tree/catalyst for Xcode 14.1. ./catalyst.sh starts with this now:
Libssh2 version: 1.10.0 (Automatically detected)
OpenSSL version: 1.1.1s (Automatically detected)
SDK version: 16.1 (Automatically detected)
Xcode version: 14.1 (Automatically detected)And ends like this:
lipo -create ${TMPDIR}iSSH2/libssh2-1.10.0/MacOSX_10.15-x86_64/install/lib/libssh2.a \
${TMPDIR}iSSH2/libssh2-1.10.0/iPhoneOS_16.1-arm64/install/lib/libssh2.a \
${TMPDIR}iSSH2/libssh2-1.10.0/iPhoneOS_16.1-arm64e/install/lib/libssh2.a -output ./libssh2_iphoneos/lib/libssh2.a
Architectures in the fat file: ./libssh2_iphoneos/lib/libssh2.a are: x86_64 arm64 arm64e
lipo -create ${TMPDIR}iSSH2/openssl-1.1.1s/MacOSX_10.15-x86_64/install/lib/libcrypto.a \
${TMPDIR}iSSH2/openssl-1.1.1s/iPhoneOS_16.1-arm64/libcrypto.a \
${TMPDIR}iSSH2/openssl-1.1.1s/iPhoneOS_16.1-arm64e/libcrypto.a -output ./openssl_iphoneos/lib/libcrypto.a
Architectures in the fat file: ./openssl_iphoneos/lib/libcrypto.a are: x86_64 arm64 arm64e
lipo -create ${TMPDIR}iSSH2/openssl-1.1.1s/MacOSX_10.15-x86_64/install/lib/libssl.a \
${TMPDIR}iSSH2/openssl-1.1.1s/iPhoneOS_16.1-arm64/libssl.a \
${TMPDIR}iSSH2/openssl-1.1.1s/iPhoneOS_16.1-arm64e/libssl.a -output ./openssl_iphoneos/lib/libssl.a
Architectures in the fat file: ./openssl_iphoneos/lib/libssl.a are: x86_64 arm64 arm64eThe x86_64 architecture in that fat file is for Mac Catalyst.
Jan-E
commented
1 year ago I ran the scripts for Catalyst and normal iOS just after the release of OpenSSL 1.1.1u. When I returned to my mail the release announcement of LibSSH2 1.11.0 arrived. As a test I also compiled this one. No errors on compilation, but I did not test it thoroughly yet. Please share any experience in this new issue: https://github.com/Frugghi/iSSH2/issues/19
Hi, I'm trying to compile for Catalyst, and the way I try doing that is through making an .xcframework. So I tried the following:
But that gave me: "error: binaries with multiple platforms are not supported 'libssh2_iphoneos/lib/libssh2.a'"
So I compiled x86_64 and arm64+arm64e separately
but that gave me a new error: "error: unable to find any architecture information in the binary at 'libssh2_iphoneos_arm64_arm64e/lib/libssh2.a'"
Even splitting arm64 and arm64e into binaries for just them, or only include arm64 or arm64e themselves, gave me the same error. Why won't it find architecture information?
Not including arm64 and arm64e at all, however, gave me success: "xcframework successfully written out to: libssh2.xcframework"
But obviously, I'd like a framework with arm64 and arm64e support. :-)
Any input on how to proceed from here? Would be really good to be able to build this for Mac Catalyst