ANEs containing Swift 5 cause crash on app launch.

tuarua commented 4 years ago

Problem Description

It can be required to compile an ANE containing both ObjectiveC and Swift in a static library. This static library is the main file used as the native library for the ANE.

Firebase is such a framework which doesn't allow housing inside a separate Dynamic Library. It must be inside a static library. This has been working fine in Xcode 9,10 and AIR 32/33 for past 2 years.

Using Xcode 11 and AIR beta 33.1.0.16 this is now causing the app to crash on launch. This happens in the most basic ANE. In fact, simply adding a single Swift Class causes the issue.

Which AIR SDK did you use?

33.1.0.16 [BETA]

Describe the affected environment:

AIR iOS

Can you reproduce it yourself, or did you get reports from other users? If so, how many users are affected (percentage)?

Yes 100%

Steps to Reproduce

Download the sample projects from https://github.com/tuarua/Swift-IOS-ANE/releases/download/4.2.0/SwiftStaticAIRIssue.zip

A README is provided which gives further details on replicating the issue, a version in Swift 4.2 which doesn't exhibit the issue and a symbolicated crash log.

Known Workarounds

None

tuarua commented 4 years ago

Problem remains in release 33.1.1.50.

tuarua commented 4 years ago

Problem remains in release 33.1.1.98.

@ajwfrost Is there a plan to look into this?

There was a brief mention in one of the 33.1.1.x release notes about looking into Swift based ANEs but it didn't seem to carry forward. I don't know if it was in general or if it meant this issue.

ajwfrost commented 4 years ago

@tuarua Hi - yes we had had a quick look at this, the failure is happening while the system is loading the Swift dylib files so this may be a tool chain issue.. Will see if we can get some more focus on this issue..!

Thanks

tuarua commented 4 years ago

Thanks

marchbold commented 4 years ago

Any news on this one?

urthling commented 4 years ago

Any news??

ajwfrost commented 4 years ago

We'd done some investigations into this but without success and it got parked - let me bump it up the list and see if one of the guys can focus on this.

tuarua commented 4 years ago

It would be great to get some movement on this. My Firebase ANE has been stuck for almost a year because of this. It's close to becoming obsolete.

marchbold commented 4 years ago

Agreed and not just firebase but several of ours are waiting for this...

tuarua commented 4 years ago

@marchbold out of interest are these ANEs which you've written in Swift instead of ObjC? - or is it that you are linking to 3rd party libs that are themselves in Swift?

marchbold commented 4 years ago

@tuarua obviously any code we control we have written (or rewritten) to work in air, the issue is with 3rd party sdks we don't control and don't have access to the source.

urthling commented 4 years ago

@ajwfrost I'm almost terrified to ask! Is there anything you can share per chance?

urthling commented 4 years ago

Please be better than Adobe were @ this bit -- I think you've gotten off to a great start and loving the work you guys have done! But us little guys need some support.

urthling commented 4 years ago

@ajwfrost I understand a release is imminent, can you please let us know if there will be a fix for this issue?

urthling commented 4 years ago

Reminder that this error happens always:

ld: library not found for -lclang_rt.ios
ld: library not found for -lclang_rt.ios
Compilation failed while executing : ld64

We need to update the SDK by copying libclang_rt.ios.a into: [AIR_SDK]/lib/aot/lib

Distriqt offer the solution here: https://github.com/distriqt/ANE-CameraRollExtended/issues/126

ajwfrost commented 4 years ago

Hi

Re the libclang_rt.ios library - I am trying to get confirmation from our OSS legal folk to ensure we are able to redistribute this; once they give the go-ahead, we can add it into the SDK/toolchain.

In terms of the underlying issue with Swift 5 here, I'm afraid we've not made any headway for the release which is about to come out, but this is definitely one of the top issues we will be focusing on for the next release. Apologies for the delays on this one, it's a very strange problem to get to the bottom of..

thanks

Andrew

urthling commented 4 years ago

@ajwfrost thank you for the update much appreciated. The fact this is now a #1 issue is really fantastic, I know I speak for everyone waiting on this, thank you.

When you say next release, if you could give us a sense of a timeframe it would really help. I know it's super hard to estimate time on an issue you haven't gotten your hands around yet but can you at least say weeks vs months?

marchbold commented 4 years ago

Just note the clang lib issue is reported here: #113 and is separate from this issue.

tuarua commented 4 years ago

but this is definitely one of the top issues we will be focusing on for the next release.

Please bear in mind Xcode 12 + iOS 14 will be out in September so any fix should also work with that toolchain. Indeed it's important in general that AIR should fast follow Android iOS base SDKs soon after their major releases.

ajwfrost commented 4 years ago

I think that it should be possible to resolve this issue using the following steps:

Add an extra rpath to the platform options for the ANE, of /usr/lib/swift
Don't include the Swift dylib files in the Frameworks when packaging with ADT
To avoid link errors you'll need the dylib files in the SDK stub folder (or the .tbd files perhaps)

This should mean that it picks up the system Swift libraries and uses those, rather than embedding the ones from the iPhone SDK which are then resulting in a call to 'abort' for some reason.

@tuarua if it will help we can attach the ADT build settings/scripts used on your 'starter_project_static_513' folder; it's running now on an iPhone whereas before it was crashing with the same log as you'd shown... or if you have a more complex example we can try that perhaps, to confirm that things actually work. At least now it's no longer crashing for us...

tuarua commented 4 years ago

I'll give those steps a go tomorrow. Sounds promising that it's not crashing!

tuarua commented 4 years ago

Hi Andrew, I've done some testing today.

Here is the project I have been testing. It is a more complete version of 513. There is a folder with some additions needed for the AIR SDK, example project, the compiled ANE + src and Xcode project.

https://github.com/tuarua/Swift-IOS-ANE/releases/download/4.2.0/starter_project_static_524.zip

Findings.

The changes you suggested means I got the app running on my iOS 13.6 and 12.4.6 Devices. Adding /usr/lib/swiftand removing the dylibs makes sense as Apple are, from 12.2 awards, packaging ABI-stable Swift on the iOS device itself.

I found the following info useful on what ld64 is now doing with Swift 5 https://milen.me/writings/apple-link-magic-swift-runtime/

The fly in the ointment

If we target say iOS 11, we would be expected to package the swift dylibs in the Frameworks folder for the App Store. adt slices these and places in SwiftSupport (?) when building an App Store app.

????

The interesting thing is that the original problem is only happening when have a static Swift library/ .a . I have no problems including these Frameworks/swift.dylibs if only ever using dynamic Swift Frameworks. It's almost like the linker is statically linking against the Frameworks/swift.dylibs when it shouldn't 🤔 As you say the .tbd are now used along with new libswiftCompatibilityDynamicReplacements.a & libswiftCompatibility50.a

Summary

Definitely great can now run on device and target 12.2 and above. Still an open question what happens with the App Store and will the lack of Swift dylibs packaged mean it would fail submission.

ajwfrost commented 4 years ago

Great, glad it's got a little further. I am wondering how it will work when deploying an app on an older (iOS11) device if this doesn't have the libraries: perhaps though, we can include these dylibs in the package but still set the rpath options up so that it would by default load in the ones from the OS if running on iOS12.2+... but then, perhaps this would still fail on the iOS11 platform. We should be able to check that. It is a little scary to see how much of this relies on 'magic' - both within the ld64 per your link above, and within the ADT code which as you say is doing all sorts of special things with Swift libraries (which is where our initial suspicions lay... but the 5.0 Swift libraries look like they should be treated the same way as 4.2...)

thanks

tuarua commented 4 years ago

Magic is the word. I remember when Swift went from 4 to 4.2 the signing broke. I helped the Adobe dev at the time track down the problem. There was a value needed to be read from the Swift dylib and was 104 hex characters ahead of the the team ID instead of 68 like before

Fun times :)

see: IPASigner > CODE_DIR_HEADERS_4_2 = 104

airsdk / Adobe-Runtime-Support