search

airsdk / Adobe-Runtime-Support

Report, track and discuss issues in Adobe AIR. Monitored by Adobe - and HARMAN - and maintained by the AIR community.
203 stars 11 forks source link

iOS ANE with Frameworks that include swiftmodules - packaging iPA fails #2685

Closed mateo-kozomara closed 11 months ago

mateo-kozomara commented 1 year ago

Problem Description

I am trying to use an ANE that uses frameworks with swift code and swiftmodules (for example, new Vungle SDK 7.0.0.)

I am able to package the ANE, but when trying to package the iOS ipa that uses the ANE i get a bunch of warnings and errors:

     [java] ld: warning: Could not find or use auto-linked library 'swiftCoreGraphics'
     [java] ld: warning: Could not find or use auto-linked library 'swiftSwiftOnoneSupport'
     [java] ld: warning: Could not find or use auto-linked library 'swiftUIKit'
     [java] ld: warning: Could not find or use auto-linked library 'swiftCompatibilityDynamicReplacements'
     [java] ld: warning: Could not find or use auto-linked library 'swiftAVFoundation'
     [java] ld: warning: Could not find or use auto-linked library 'swiftDarwin'
     [java] ld: warning: Could not find or use auto-linked library 'swiftQuartzCore'
     [java] ld: warning: Could not find or use auto-linked library 'swiftUniformTypeIdentifiers'
     [java] ld: warning: Could not find or use auto-linked library 'swiftCompatibilityConcurrency'
     [java] ld: warning: Could not find or use auto-linked library 'swiftMetal'
     [java] ld: warning: Could not find or use auto-linked library 'swiftNetwork'
     [java] ld: warning: Could not find or use auto-linked library 'swift_Concurrency'
     [java] ld: warning: Could not find or use auto-linked library 'swiftos'
     [java] ld: warning: Could not find or use auto-linked library 'swiftsimd'
     [java] ld: warning: Could not find or use auto-linked library 'swiftCore'
     [java] ld: warning: Could not find or use auto-linked library 'swiftCompatibility50'
     [java] ld: warning: Could not find or use auto-linked library 'swiftCoreImage'
     [java] ld: warning: Could not find or use auto-linked library 'swiftFileProvider'
     [java] ld: warning: Could not find or use auto-linked library 'swiftCoreMedia'
     [java] ld: warning: Could not find or use auto-linked library 'swiftObjectiveC'
     [java] ld: warning: Could not find or use auto-linked library 'swiftCoreMIDI'
     [java] ld: warning: Could not find or use auto-linked library 'swiftFoundation'
     [java] ld: warning: Could not find or use auto-linked library 'swiftCoreAudio'
     [java] ld: warning: Could not find or use auto-linked library 'swiftCompatibility51'
     [java] ld: warning: Could not find or use auto-linked library 'swiftCoreData'
     [java] ld: warning: Could not find or use auto-linked library 'swiftDispatch'
     [java] ld: warning: Could not find or use auto-linked library 'swiftCoreFoundation'
     [java] ld: warning: Could not find or use auto-linked library 'swiftDataDetection'
     [java] Undefined symbols for architecture arm64:
     [java]   "_$s12CoreGraphics7CGFloatVMn", referenced from:
     [java]       _symbolic _____ 12CoreGraphics7CGFloatV in VungleAdsSDK(sdk.pb.o)
     [java]   "_$s15_ObjectiveCTypes01_A11CBridgeablePTl", referenced from:
     [java]       l_got.$s15_ObjectiveCTypes01_A11CBridgeablePTl in VungleAdsSDK(sdk.pb.o)
     [java]       l_got.$s15_ObjectiveCTypes01_A11CBridgeablePTl in UnityAds(UnityAds-arm64-master.o)
     [java]   "_$s16FloatLiteralTypes013ExpressibleByaB0PTl", referenced from:
     [java]       l_got.$s16FloatLiteralTypes013ExpressibleByaB0PTl in VungleAdsSDK(Google_Protobuf_Value+Extensions.o)
     [java]       l_got.$s16FloatLiteralTypes013ExpressibleByaB0PTl in VungleAdsSDK(Google_Protobuf_Duration+Extensions.o)
     [java]       l_got.$s16FloatLiteralTypes013ExpressibleByaB0PTl in VungleAdsSDK(Google_Protobuf_Wrappers+Extensions.o)
     [java]   "_$s17StringLiteralTypes013ExpressibleByaB0PTl", referenced from:
     [java]       l_got.$s17StringLiteralTypes013ExpressibleByaB0PTl in VungleAdsSDK(Google_Protobuf_Value+Extensions.o)
     [java]       l_got.$s17StringLiteralTypes013ExpressibleByaB0PTl in VungleAdsSDK(Google_Protobuf_Wrappers+Extensions.o)
     [java]   "_$s18BooleanLiteralTypes013ExpressibleByaB0PTl", referenced from:
     [java]       l_got.$s18BooleanLiteralTypes013ExpressibleByaB0PTl in VungleAdsSDK(Google_Protobuf_Value+Extensions.o)
     [java]       l_got.$s18BooleanLiteralTypes013ExpressibleByaB0PTl in VungleAdsSDK(Google_Protobuf_Wrappers+Extensions.o)
     [java]   "_$s18IntegerLiteralTypes013ExpressibleByaB0PTl", referenced from:
     [java]       l_got.$s18IntegerLiteralTypes013ExpressibleByaB0PTl in VungleAdsSDK(Google_Protobuf_Value+Extensions.o)
     [java]       l_got.$s18IntegerLiteralTypes013ExpressibleByaB0PTl in VungleAdsSDK(Google_Protobuf_Wrappers+Extensions.o)
     [java]   "_$s19ArrayLiteralElements013ExpressibleByaB0PTl", referenced from:
     [java]       l_got.$s19ArrayLiteralElements013ExpressibleByaB0PTl in VungleAdsSDK(SimpleExtensionMap.o)
     [java]       l_got.$s19ArrayLiteralElements013ExpressibleByaB0PTl in VungleAdsSDK(Google_Protobuf_ListValue+Extensions.o)
     [java]   "_$s24UnicodeScalarLiteralTypes013ExpressibleByabC0PTl", referenced from:
     [java]       l_got.$s24UnicodeScalarLiteralTypes013ExpressibleByabC0PTl in VungleAdsSDK(Google_Protobuf_Value+Extensions.o)
     [java]       l_got.$s24UnicodeScalarLiteralTypes013ExpressibleByabC0PTl in VungleAdsSDK(Google_Protobuf_Wrappers+Extensions.o)
     [java]   "_$s34ExtendedGraphemeClusterLiteralTypes013ExpressibleByabcD0PTl", referenced from:
     [java]       l_got.$s34ExtendedGraphemeClusterLiteralTypes013ExpressibleByabcD0PTl in VungleAdsSDK(Google_Protobuf_Value+Extensions.o)
     [java]       l_got.$s34ExtendedGraphemeClusterLiteralTypes013ExpressibleByabcD0PTl in VungleAdsSDK(Google_Protobuf_Wrappers+Extensions.o)

In the native iOS project I am packaging a static library target. I tried including libSwiftCompat.a from AIR SDK in ANE and when packaging the ipa, but it didnt help.

In platform-options.xml for iOS i have this option enabled: <option>-rpath /usr/lib/swift</option>as well as in library search paths in XCode native project.

Am i maybe missing something? Here is the adt.log from building the ipa.

I am using AIR SDK 50.2.2.6 and XCode 14.1, on a M1 Mac with MacOS Ventura 13.2.1. I am testing on a iOS device running iOS 15.7.

Steps to Reproduce

Package an iOS ANE that includes a Framework with swift code and swiftmodules (for example Vungle 7.0.0) Try packaging the iOS app that uses that ANE. Packaging fails with warnings (Could not find or use auto-linked library) and errors (Undefined symbols for architecture arm64)

Known Workarounds

No known workarounds.

marchbold commented 1 year ago

Try adding 

        <option>-lSwiftCompat</option>
        <option>-rpath /usr/lib/swift</option>

To your linker options in the platform.xml.

mateo-kozomara commented 1 year ago

Thanks @marchbold, I tried adding that but it doesn't seem to make a difference. I also tried adding it -lSwiftCompat to 'Other Linker Flags' in XCode project.

Is there maybe a way i can explicitly include/package those missing swift libraries in the ane? I tried including the dylibs, for example libswiftAVFoundation.dylib in the ANE, but also in the dynamic Frameworks folder that is packaged with the IPA, but it didnt help. I get 'Negative offset' errors when signing the IPA.

mateo-kozomara commented 1 year ago

Finally made some progress.

Looks like the issue is related to setting the -platformsdk when packaging the ipa. I tried with XCode 14.3.1 and XCode 14.2 but both of them result in missing auto linked swift libraries.

Also, i had to copy these libraries from XCode Toolchain to AIRSDK/lib/aot/lib folder:

libswiftCompatibility50.a
libswiftCompatibility51.a
libswiftCompatibilityConcurrency.a
libswiftCompatibilityDynamicReplacements.a

and add this when packaging the ANE in platform.xml

<option>-lSwiftCompat</option>
<option>-rpath /usr/lib/swift</option>

after this the ipa packages successfully and looks like the code works.

But it kinda sucks we can no longer specify -platformsdk during packaging.

Is this something that will be resolved in the future?

ajwfrost commented 1 year ago

I had to copy these libraries from XCode Toolchain to AIRSDK/lib/aot/lib folder

Those libraries are provided (combined) in the SDK under lib/aot/lib/libSwiftCompat.a so the -lSwiftCompat should pick those up okay.

But it kinda sucks we can no longer specify -platformsdk during packaging.

You can still use that .. but I'm wondering whether this may be related to a priority issue that we have just resolved, whereby a configuration file may override a command-line argument if it's placed immediately after the xml descriptor file (if it's placed at the end, it seems to be okay). Although, your earlier adt.log file shows it was picking up a platform SDK at /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS16.1.sdk so that does seem to have worked (assuming this was the value you intended to use! adt hadn't been passed a platform SDK on the command line...)

Can you confirm what you had to change around this platform SDK option to get it to work?

thanks

mateo-kozomara commented 1 year ago

@ajwfrost thank you, yes i can verify that lib/aot/lib/libSwiftCompat.a was all that was necessary, I did not need to copy the other swift compatibility to air sdk.

However, specifying platformsdk does still not work. I tried changing the order of the arguments, but I still get missing auto linked swift libraries

marchbold commented 1 year ago

Could be an update to the swift version in the latest sdk?

ajwfrost commented 1 year ago

I'm not actually sure what error you're getting now, or what you're expecting from the platform sdk setting..?

still get missing auto linked swift libraries

Do you mean, per the original report, where you have:

     [java] ld: warning: Could not find or use auto-linked library 'swiftCoreGraphics'
etc

Which AIR SDK are you using? We added these into the stub folder recently, so you should find these automatically linking: aot/lib/stub/libswiftCoreGraphics.dylib etc

thanks

mateo-kozomara commented 1 year ago

@ajwfrost yes, using platformsdk option produces 'Could not find or use auto-linked library' errors for swift libraries. I am using AIR SDK 50.2.2.6 and setting the platformsdk flag pointing to XCode 14.2 iOS SDK, so there should be no differences with the swift libraries inside AIR SDK, since release notes state XCode 14.2 was used to create the AIR SDK.

ajwfrost commented 1 year ago

Thanks -> so just wondering, how about if you remove/rename those libswift*.dylib files from the AIR SDK? Maybe there's a conflict there.

Also just to check, it sounds like yesterday you had got the IPA to build? so I'm not sure what the difference is between that situation and the current one?

after this the ipa packages successfully and looks like the code works.

thanks

mateo-kozomara commented 1 year ago

@ajwfrost yep, i already removed the extra libswift*.dylib from AIR SDK, so shouldnt be related to that.

Yes, i got the iPA to build, but without setting platformsdk option when packaging. With platformsdk option the build fails with missing auto-linked swift libraries.

mateo-kozomara commented 1 year ago

Okay, so even though im not getting swift auto-link errors, looks like there are still issues... After including a framework that has .swiftmodules inside (for example UnityAds SDK v4.7.1), the app crashes on startup with error below, even though Foundation is included, along with lSwiftCompat option in platform.xml.

Any tips on how to avoid this error? It kinda looks like it might have to do with swift versions mismatch or something like that?

ASI found [dyld] (sensitive) 'Symbol not found: (_$s10Foundation10URLRequestV10httpMethodSSSgvs)
  Referenced from: '/private/var/containers/Bundle/Application/F33D2F58-9863-4DFB-B814-CA53FC00BCB8/***.app/***'
  Expected in: '/System/Library/Frameworks/Foundation.framework/Foundation''
ajwfrost commented 1 year ago

So that particular symbol: Foundation.URLRequest.httpMethod is mentioned in the Foundation framework, at least in the .tbd file in the SDK: '$ld$previous$/usr/lib/swift/libswiftFoundation.dylib$1.0.0$2$12.2$16.0$_$s10Foundation10URLRequestV10httpMethodSSSgvM$' and the dylib file there seems to indicate that the symbols are in the .dylib if it's on a 32-bit device but in the Foundation framework if it's a 64-bit one.

You say that "even though Foundation is included" - how are you including this? or do you just mean it's included in the list of frameworks to link against? And just to double-check, it is a newish 64-bit device you're running on?

thanks

mateo-kozomara commented 1 year ago

Thanks @ajwfrost thats very interesting, so it could be that somehow its not picking the correct Foundation library.

The device im testing on is a iPhone Xs Max with iOS 15.7.

I tried including the Foundation framework with -lswiftFoundation linker option in platform.xml and I also added that to XCode projects 'Other Linker Flags'.

I also tried inlcuding libswiftFoundation.tbd in 'Link Binary With Libraries' section. And I also have -framework Foundation in platform.xml as well.

None of these seemed to have any effect and 'symbol not found' error persists

mateo-kozomara commented 1 year ago

Did anyone manage to successfully build an iOS ANE that uses a framework which includes swiftmodules inside? (for example UnityAds SDK v4.7.1)

sweetnitro commented 1 year ago

Hello,

We might have the same issue with Ironsource distriqt ANE for both mediations that uses swift foundation (Unity & Chartboost).

Just in case it's related herewith the link: https://github.com/distriqt/ANE-IronSource/issues/54

Regards

marchbold commented 1 year ago

We do have extensions that are using swiftmodules successfully but do seem to be having an issue with UnityAds in particular. We are investigating but haven't isolated the issue as yet.

mateo-kozomara commented 1 year ago

@marchbold the issue does not occur only with UnityAds, its also occurs with VungleSDK v7.0.0 for example. Im sure there are other SDKs affected by this issue as well

Just so there is no confusion, i mean this error:


ASI found [dyld] (sensitive) 'Symbol not found: (_$s10Foundation10URLRequestVMn)
  Referenced from: '/private/var/containers/Bundle/Application/684E7091-A305-41B8-8B6D-04BE1712E98D/xxx.app/xxx'
  Expected in: '/System/Library/Frameworks/Foundation.framework/Foundation''
takazawa-gg commented 1 year ago

I have the same problem.

Devices running iOS 16.6 or 16.5.1 can launch the app. Devices running iOS 15.x, 12.5.7, etc. will crash on startup.

[0,0],"type":"EXC_CRASH","signal":"SIGABRT"},
"termination" : {
  "code":4,
  "flags":518,
  "namespace":"DYLD",
  "indicator":"Symbol missing",
  "details":["(terminated at launch; ignore backtrace)"],
  "reasons":["Symbol not found: (_$s10Foundation10URLRequestVMn)",
  "Referenced from: '\/Volumes\/VOLUME\/*\/air.app\/air'",
  "Expected in: '\/System\/Library\/Frameworks\/Foundation.framework\/Foundation'"]
},

No workaround has yet been found.

ajwfrost commented 1 year ago

I'm wondering here if there's a change in the location of that symbol, which may be causing problems when running on a device. The symbol itself (_$s10Foundation10URLRequestVMn) is mentioned both in the Foundation framework, and in the libswiftFoundation archive (at least, when looking at the latest SDK..). So when you link, it might be that the binary is assuming that the symbol should be picked up from the Foundation framework, and probably then that would work for the newer devices but perhaps the older devices have a Foundation framework that is missing this symbol.

We can try to do a few tests to see whether we can get it to pick up the symbol from the Swift dylib instead (not 100% sure whether that would result in a fix, but it may help..). Probably the best way to proceed is if we upload a new "Foundation" library here, that can be put into the stubs folder, and then the app rebuilt...

I would also be interested in whether this is just an issue with apps built on Windows, or whether it's the same for apps built on macOS...

thanks

ajwfrost commented 1 year ago

Hi

Updated adt.jar file is available from the below comment - if anyone is able to see whether this helps to avoid the crashes due to symbols being expected in the Foundation framework, it would be helpful..

thanks Andrew

https://github.com/airsdk/Adobe-Runtime-Support/issues/2768#issuecomment-1682626000

marchbold commented 1 year ago

@ajwfrost Sorry for the delay in getting back to you on this one.

We have been able to confirm that an application running on iOS 12.5.5 built with AIR 50.2.3.5 was crashing with 

Exception Type:  EXC_CRASH (SIGABRT)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Exception Note:  EXC_CORPSE_NOTIFY
Termination Description: DYLD, Symbol not found: _$s10Foundation10CocoaErrorV4CodeVAA01_cD8ProtocolAAMc

Updating with the adt.jar from the comment resolved the issue!

It is also working with this updated adt on more recent versions including iOS 16.6

marchbold commented 11 months ago

Resolved in 50.2.3.6