xamarin / xamarin-macios

.NET for iOS, Mac Catalyst, macOS, and tvOS provide open-source bindings of the Apple SDKs for use with .NET managed languages such as C#
Other
2.43k stars 507 forks source link

Invoking async swift code in an iOS binding library is crashing with an objective C encoding error #18562

Closed jerrgreen closed 3 months ago

jerrgreen commented 1 year ago

Redirected here from MAUI

I am trying to integrate an iOS binding library with a swift library that uses asynchronous methods. When I hit a method that is synchronous, the method executes successfully. When I try to invoke any asynchronous method, the app crashes. The last exception that I see in the logs is "Microsoft.iOS: Unsupported type encoding: "NSString"16@?v@?@"NSString"24".

Referencing the Binding details for Xamarin,, I'd expect this to be fully supported, but I cannot find a way past this error.

I've been able to reproduce this on both the latest versions of dotnet 6 and dotnet 7.

Steps to Reproduce

  1. Have a swift library that utilizes async code
  2. Create a binding library to that swift library
  3. Attempt to invoke the code from your app

Expected Behavior

The swift method executes successfully

Actual Behavior

The application crashes

Environment

Version information ``` Visual Studio Enterprise 2022 for Mac Version 17.5.6 (build 3) Installation UUID: 19b400c3-ed20-4e3d-ad39-9d4910716a49 Runtime .NET 7.0.1 (64-bit) Architecture: Arm64 Microsoft.macOS.Sdk 12.3.2372; git-rev-head:754abbf6a3563f6267e5717ae832b4ac25b1f2fb; git-branch:release/7.0.1xx-xcode13.3 Roslyn (Language Service) 4.5.0-3.23056.2+97881342e427ff5cdcba8f12b12ff8e6f3564431 NuGet Version: 6.4.0.117 .NET SDK (Arm64) SDK: /usr/local/share/dotnet/sdk/7.0.306/Sdks SDK Versions: 7.0.306 7.0.302 6.0.412 6.0.408 MSBuild SDKs: /Applications/Visual Studio.app/Contents/MonoBundle/MSBuild/Current/bin/Sdks .NET SDK (x64) SDK Versions: 8.0.100-preview.6.23309.2 7.0.306 6.0.408 .NET Runtime (Arm64) Runtime: /usr/local/share/dotnet/dotnet Runtime Versions: 7.0.9 7.0.5 6.0.20 6.0.16 .NET Runtime (x64) Runtime: /usr/local/share/dotnet/x64/dotnet Runtime Versions: 8.0.0-preview.6.23307.4 8.0.0-preview.6.23280.5 7.0.9 6.0.16 Xamarin.Profiler '/Applications/Xamarin Profiler.app' not found Updater Version: 11 Xamarin.Android Not Installed Microsoft Build of OpenJDK Java SDK: /Library/Java/JavaVirtualMachines/microsoft-11.jdk 11.0.16.1 Android Designer EPL code available here: https://github.com/xamarin/AndroidDesigner.EPL Eclipse Temurin JDK Java SDK: Not Found Android SDK Manager Version: 17.5.0.33 Hash: f0c0c52 Branch: remotes/origin/d17-5~2 Build date: 2023-05-18 17:57:54 UTC Android Device Manager Version: 0.0.0.1245 Hash: 7f8a990 Branch: 7f8a990 Build date: 2023-05-18 17:57:55 UTC Apple Developer Tools Xcode: 14.3.1 21815 Build: 14E300c Xamarin.Mac Not Installed Xamarin.iOS Not Installed Xamarin Designer Version: 17.5.3.47 Hash: e8b5d371c3 Branch: remotes/origin/d17-5 Build date: 2023-05-18 17:57:49 UTC Build Information Release ID: 1705060003 Git revision: 6472fea49fa01feff06feefde4e358c02df42934 Build date: 2023-05-18 17:56:09+00 Build branch: release-17.5 Build lane: release-17.5 Operating System Mac OS X 13.4.1 Darwin 22.5.0 Darwin Kernel Version 22.5.0 Thu Jun 8 22:21:34 PDT 2023 root:xnu-8796.121.3~7/RELEASE_ARM64_T8112 arm64 ```

Build Logs

Logs ``` 2023-07-17 11:59:48.048 MauiAppForAsync[758:186937] Printing first message 2023-07-17 11:59:48.051 MauiAppForAsync[758:186937] ECHO: test 2023-07-17 11:59:48.051 MauiAppForAsync[758:186937] Printing second message 2023-07-17 11:59:48.053 MauiAppForAsync[758:186937] Microsoft.iOS: Unsupported type encoding: "NSString"16@?24 test ================================================================= Native Crash Reporting ================================================================= Got a SIGABRT while executing native code. This usually indicates a fatal error in the mono runtime or one of the native libraries used by your application. ================================================================= ================================================================= Native stacktrace: ================================================================= 0x1010a4b18 - /private/var/containers/Bundle/Application/18836C3D-F2F9-401F-82F9-90E023402D25/PublicStaging.app/MauiAppForAsync : _ZNK3icu6number23NumberFormatterSettingsINS0_24LocalizedNumberFormatterEE10toSkeletonER10UErrorCode 0x10108fc14 - /private/var/containers/Bundle/Application/18836C3D-F2F9-401F-82F9-90E023402D25/PublicStaging.app/MauiAppForAsync : _ZNK3icu6number23NumberFormatterSettingsINS0_24LocalizedNumberFormatterEE10toSkeletonER10UErrorCode 0x1011d7818 - /private/var/containers/Bundle/Application/18836C3D-F2F9-401F-82F9-90E023402D25/PublicStaging.app/MauiAppForAsync : _ZNK3icu6number23NumberFormatterSettingsINS0_24LocalizedNumberFormatterEE10toSkeletonER10UErrorCode 0x1010a432c - /private/var/containers/Bundle/Application/18836C3D-F2F9-401F-82F9-90E023402D25/PublicStaging.app/MauiAppForAsync : _ZNK3icu6number23NumberFormatterSettingsINS0_24LocalizedNumberFormatterEE10toSkeletonER10UErrorCode 0x219314a90 - /usr/lib/system/libsystem_platform.dylib : 0x2193b01ac - /usr/lib/system/libsystem_pthread.dylib : pthread_kill 0x1d2998c8c - /usr/lib/system/libsystem_c.dylib : abort 0x100dae458 - /private/var/containers/Bundle/Application/18836C3D-F2F9-401F-82F9-90E023402D25/PublicStaging.app/MauiAppForAsync : xamarin_get_block_descriptor 0x100db2ed8 - /private/var/containers/Bundle/Application/18836C3D-F2F9-401F-82F9-90E023402D25/PublicStaging.app/MauiAppForAsync : xamarin_find_protocol_wrapper_type 0x100db7228 - /private/var/containers/Bundle/Application/18836C3D-F2F9-401F-82F9-90E023402D25/PublicStaging.app/MauiAppForAsync : _Z9__isctypeim 0x100dc6060 - /private/var/containers/Bundle/Application/18836C3D-F2F9-401F-82F9-90E023402D25/PublicStaging.app/MauiAppForAsync : _ZN16XamarinCallState3selEv 0x100d8142c - /private/var/containers/Bundle/Application/18836C3D-F2F9-401F-82F9-90E023402D25/PublicStaging.app/MauiAppForAsync : 0x1010b2de8 - /private/var/containers/Bundle/Application/18836C3D-F2F9-401F-82F9-90E023402D25/PublicStaging.app/MauiAppForAsync : _ZNK3icu6number23NumberFormatterSettingsINS0_24LocalizedNumberFormatterEE10toSkeletonER10UErrorCode 0x1010a856c - /private/var/containers/Bundle/Application/18836C3D-F2F9-401F-82F9-90E023402D25/PublicStaging.app/MauiAppForAsync : _ZNK3icu6number23NumberFormatterSettingsINS0_24LocalizedNumberFormatterEE10toSkeletonER10UErrorCode 0x1010a6044 - /private/var/containers/Bundle/Application/18836C3D-F2F9-401F-82F9-90E023402D25/PublicStaging.app/MauiAppForAsync : _ZNK3icu6number23NumberFormatterSettingsINS0_24LocalizedNumberFormatterEE10toSkeletonER10UErrorCode 0x10107704c - /private/var/containers/Bundle/Application/18836C3D-F2F9-401F-82F9-90E023402D25/PublicStaging.app/MauiAppForAsync : _ZNK3icu6number23NumberFormatterSettingsINS0_24LocalizedNumberFormatterEE10toSkeletonER10UErrorCode 0x100fc6714 - /private/var/containers/Bundle/Application/18836C3D-F2F9-401F-82F9-90E023402D25/PublicStaging.app/MauiAppForAsync : _ZNK3icu6number23NumberFormatterSettingsINS0_24LocalizedNumberFormatterEE10toSkeletonER10UErrorCode 0x100fdad44 - /private/var/containers/Bundle/Application/18836C3D-F2F9-401F-82F9-90E023402D25/PublicStaging.app/MauiAppForAsync : _ZNK3icu6number23NumberFormatterSettingsINS0_24LocalizedNumberFormatterEE10toSkeletonER10UErrorCode 0x100fdaad4 - /private/var/containers/Bundle/Application/18836C3D-F2F9-401F-82F9-90E023402D25/PublicStaging.app/MauiAppForAsync : _ZNK3icu6number23NumberFormatterSettingsINS0_24LocalizedNumberFormatterEE10toSkeletonER10UErrorCode 0x2193aa6cc - /usr/lib/system/libsystem_pthread.dylib : _pthread_start 0x2193a9ba4 - /usr/lib/system/libsystem_pthread.dylib : thread_start ================================================================= Basic Fault Address Reporting ================================================================= Memory around native instruction pointer (0x208c93160):0x208c93150 ff 0f 5f d6 c0 03 5f d6 10 29 80 d2 01 10 00 d4 .._..._..)...... 0x208c93160 03 01 00 54 7f 23 03 d5 fd 7b bf a9 fd 03 00 91 ...T.#...{...... 0x208c93170 8e ed ff 97 bf 03 00 91 fd 7b c1 a8 ff 0f 5f d6 .........{...._. 0x208c93180 c0 03 5f d6 7f 23 03 d5 fd 7b bf a9 fd 03 00 91 .._..#...{...... ================================================================= Managed Stacktrace: ================================================================= at <0xffffffff> at ApiDefinitions.Messaging:void_objc_msgSend_NativeHandle_NativeHandle <0x0006e> at IosBinding.TestClass:DebugLog2WithMessage <0x00104> at IosBinding.TestClass:DebugLog2WithMessageAsync <0x00060> at d__0:MoveNext <0x000d0> at System.Runtime.CompilerServices.AsyncMethodBuilderCore:Start <0x00076> at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:Start <0x0000e> at MauiAppForAsync.Services.AsyncInvokerService:DebugLog <0x00084> at <b__0>d:MoveNext <0x0005a> at System.Runtime.CompilerServices.AsyncMethodBuilderCore:Start <0x00076> at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:Start <0x0000e> at <>c__DisplayClass1_0:b__0 <0x00070> at System.Threading.Tasks.Task`1:InnerInvoke <0x0004a> at <>c:<.cctor>b__273_0 <0x0001c> at System.Threading.ExecutionContext:RunFromThreadPoolDispatchLoop <0x00064> at System.Threading.Tasks.Task:ExecuteWithThreadLocal <0x001ee> at System.Threading.Tasks.Task:ExecuteEntryUnsafe <0x00082> at System.Threading.Tasks.Task:ExecuteFromThreadPool <0x00014> at System.Threading.ThreadPoolWorkQueue:DispatchWorkItem <0x00042> at System.Threading.ThreadPoolWorkQueue:DispatchItemWithAutoreleasePool <0x00054> at System.Threading.ThreadPoolWorkQueue:Dispatch <0x00382> at WorkerThread:WorkerThreadStart <0x00108> at StartHelper:RunWorker <0x00086> at StartHelper:Run <0x0006a> at System.Threading.Thread:StartCallback <0x00040> at System.Object:runtime_invoke_direct_void__this__ <0x00048> at <0x00000> ```

Example Project (If Possible)

Sample Repo

dalexsoto commented 1 year ago

Hello @jerrgreen sorry for the late response I was able to repro with the provided steps, would you be able to share with us the Swift library source?

Thanks!

ghost commented 1 year ago

Hi @jerrgreen. We have added the "need-info" label to this issue, which indicates that we have an open question for you before we can take further action. This issue will be closed automatically in 7 days if we do not hear back from you by then - please feel free to re-open it if you come back to this issue after that time.

jerrgreen commented 1 year ago

The library from the example is a pretty straightforward class

class TestClass {
    func DebugLog(message: String) -> String {
        print(message)
        return message
    }

    func DebugLog2(message: String) async -> String {
        do {
            try await Task.sleep(nanoseconds: 1_000_000_000)
        } catch {}

        print(message);
        return message;
    }
}
thefex commented 1 year ago

It seems to work on real device though only simulator crash

thefex commented 1 year ago

Please consider setting it as high priority issue that needs to be solved asap otherwise many important iOS API's could not be properly bound (at least for simulator..) - like StoreKit2 which makes development very difficult (simulator builds becomes unusable). Some of StoreKit2 API should be mapped due to the App Store guideliness (like give button and call manage subscriptions whenever user deletes account)

To reproduce the issue please install following package:

var response = await StoreKitAppStoreExtensions.ShowManageSubscriptionsIn(UIApplication.SharedApplication.KeyWindow.WindowScene);

On device:

On simulator:

You can get problematic .xcframework, swift project, xamarin bindings projects here: https://github.com/thefex/Xamarin.iOS.StoreKit2Sample

jerrgreen commented 1 year ago

My experience is not limited to the simulator, I am seeing these crashes on real devices

thefex commented 1 year ago

My experience is not limited to the simulator, I am seeing these crashes on real devices

Do you use xcframework? Or maybe the issue is more broad but in my case works..? I am using Xamarin Native (not MAUI) though without .NET 6

jerrgreen commented 1 year ago

Yes, we're using xcframeworks for our swift integration

rolfbjarne commented 1 year ago

I can reproduce this with the provided sample.

rolfbjarne commented 1 year ago

The problem is that Swift seems to produce Objective-C code that doesn't conform to the spec.

Using NSMethodSignature in an Xcode project also complains:

'+[NSMethodSignature signatureWithObjCTypes:]: unsupported type encoding spec '"' in '"NSString"16@?v@?@"NSString"24''

thefex commented 1 year ago

So this seems to be a bug in XCode / obj-c / swift compiler ? Are there any known workaround for such problems?

rolfbjarne commented 1 year ago

So this seems to be a bug in XCode / obj-c / swift compiler ? Are there any known workaround for such problems?

I don't really think it's a bug. They just changed stuff and never bothered to update the documentation/spec.

thefex commented 1 year ago

Hm... okey, can it be solved on Xamarin level though? Any tips how developer can still safely use those async swift methods in Xamarin.iOS?

rolfbjarne commented 1 year ago

can it be solved on Xamarin level though?

If we can figure out the format they're using, yes - so I asked on StackOverflow: https://stackoverflow.com/questions/76827926/whats-the-format-for-objc-method-description-types-for-swift-blocks

Any tips how developer can still safely use those async swift methods in Xamarin.iOS?

Try adding the following to your csproj:

<PropertyGroup>
    <MtouchExtraArgs>--require-pinvoke-wrappers=true</MtouchExtraArgs>
</PropertyGroup>
thefex commented 1 year ago

Thanks. I think that extended method format is used at least for async/await, closures blocks, function types, parameter attributes. This "< >" represents closure function and "@NSString" inside represent the single parameter that this closure takes (it's v so it does not have return type)

Do traditional non-async closure blocks (like regular handlers) can be parsed normally and there are no issues? Maybe the issue is that swift can generate perfect fine representation for normal closure but as async block is represented differently in obj-c (it has additional 'method param' -> completion handler which is kind of a closure) and it somehow has not been handled properly?

Sorry, if that does not make much sense but I am not an expert in those internals

rolfbjarne commented 1 year ago

Do traditional non-async closure blocks (like regular handlers) can be parsed normally and there are no issues?

As long as the method signature string doesn't contain ", < or >, it should be fine (unless there are more things we don't know about).

thefex commented 1 year ago

Hm... then I think the problem are more widespread than that.. and probably not solvable 'the easy way'

Those extended swift types seems to be not parsable in obj-c, and obj-c runtime somehow finds 'swift compiler generated method thunk' that calls swift method taking care about all of the internal representation..

I just wonder.. maybe the easy workaround would be creating additional obj-c methods that calls those swift obj-c compatibile methods and bind only those obj-c methods?

EDIT: At least in theory, there might be two method pointers in framework, one with method signature parsable by objc (that takes cares internally about signature mismatch) and (the one that is called by 'previous' method (with extended method signature)

EDIT2: Seems that might be the case, take a look at

(base) Przemysaws-MacBook-Pro-2:ios-arm64_x8664-simulator przemyslawraciborski$ nm -g StoreKit2.framework/StoreKit2 U $s10Foundation22_convertErrorToNSErrorySo0E0Cs0C0pF U $s8StoreKit03AppA0O23showManageSubscriptions2inySo13UIWindowSceneCtYaKFZ U $s8StoreKit03AppA0O23showManageSubscriptions2inySo13UIWindowSceneCtYaKFZTu 00000000000031bc T $s9StoreKit20a6KitAppA0C23showManageSubscriptions2inySo13UIWindowSceneCtYaKFZ 0000000000008108 D $s9StoreKit20a6KitAppA0C23showManageSubscriptions2inySo13UIWindowSceneC_tYaKFZTu

One method seems to have two function ptrs (though one is marked as U -> linked outside of .framework which I dont get completely).

jerrgreen commented 1 year ago

Any tips how developer can still safely use those async swift methods in Xamarin.iOS?

Try adding the following to your csproj:

<PropertyGroup>
    <MtouchExtraArgs>--require-pinvoke-wrappers=true</MtouchExtraArgs>
</PropertyGroup>

Good news, I added this to the property and can confirm that the async code can be invoked, so this does look like a valid work around for the issue.

thefex commented 1 year ago

@rolfbjarne Are there any docs for --require pinvoke? What it exactly does?

rolfbjarne commented 1 year ago

@rolfbjarne Are there any docs for --require pinvoke? What it exactly does?

For every P/Invoke to the Objective-C messaging functions (objc_msgSend) we create a native wrapper function and call that wrapper function instead of the objc_msgSend.

The downside is that the app will be bigger due to the extra code.

The upside is that the code will likely be slightly faster (the default is to call a generic wrapper function that works for all P/Invokes - this is slightly slower, but smaller - and this is where the bug is, in this generic wrapper functoin we need to figure out the signature of the target native method we need to call, and that's where we run into what the Swift compiler does).

marcb777 commented 1 year ago

Does "--require-pinvoke-wrappers=true" work on a "net7.0-ios" bindings library?

I'm having the same issue of apps crashing when calling async Swift bindings. The only thing that is logged is: "Microsoft.iOS: Unsupported type encoding: "NSString"16@?v@?@"NSString"24"

The --require-pinvoke-wrappers=true doesn't seem to have an impact. This page on migrating projects seems to indicate that some MtouchExtraArgs are no longer applicable: https://learn.microsoft.com/en-us/dotnet/maui/migration/apple-projects

rolfbjarne commented 1 year ago

Does "--require-pinvoke-wrappers=true" work on a "net7.0-ios" bindings library?

You need to add it to the executable project, not the bindings project.

If that doesn't work, please attach a binlog for a build with the problem.

marcb777 commented 1 year ago

Sorry if I'm being dumb, but when I enable "--require-pinvoke-wrappers=true" on the executable project, I get an exception on launch (see below). I suspect this is an unrelated issue.

Unhandled Exception:
ObjCRuntime.RuntimeException: Failed to marshal the Objective-C object 0x7fea45026ab0 (type: UIScreen). Could not find an existing managed instance for this object, nor was it possible to create a new managed instance (because the type 'CoreGraphics.CGBitmapFlags' does not have a constructor that takes one NativeHandle argument).
   at ObjCRuntime.Runtime.MissingCtor(IntPtr ptr, IntPtr klass, Type type, MissingCtorResolution resolution)
   at ObjCRuntime.Runtime.ConstructNSObject[UIScreen](IntPtr ptr, Type type, MissingCtorResolution missingCtorResolution)
   at ObjCRuntime.Runtime.GetNSObject[UIScreen](IntPtr ptr)
   at UIKit.UIScreen.get_MainScreen()
   at TestApp.AppDelegate.FinishedLaunching(UIApplication application, NSDictionary launchOptions) in /.../TestApp/AppDelegate.cs:line 13
   at UIKit.UIApplication.UIApplicationMain(Int32 argc, String[] argv, IntPtr principalClassName, IntPtr delegateClassName)
   at UIKit.UIApplication.Main(String[] args, Type principalClass, Type delegateClass)
   at Program.<Main>$(String[] args) in /.../TestApp/Main.cs:line 6
2023-08-11 07:59:31.363675+1000 TestApp[93297:13832976] Unhandled managed exception: Failed to marshal the Objective-C object 0x7fea45026ab0 (type: UIScreen). Could not find an existing managed instance for this object, nor was it possible to create a new managed instance (because the type 'CoreGraphics.CGBitmapFlags' does not have a constructor that takes one NativeHandle argument). (ObjCRuntime.RuntimeException)
   at ObjCRuntime.Runtime.MissingCtor(IntPtr ptr, IntPtr klass, Type type, MissingCtorResolution resolution)
   at ObjCRuntime.Runtime.ConstructNSObject[UIScreen](IntPtr ptr, Type type, MissingCtorResolution missingCtorResolution)
   at ObjCRuntime.Runtime.GetNSObject[UIScreen](IntPtr ptr)
   at UIKit.UIScreen.get_MainScreen()
   at TestApp.AppDelegate.FinishedLaunching(UIApplication application, NSDictionary launchOptions) in /.../TestApp/AppDelegate.cs:line 13
   at UIKit.UIApplication.UIApplicationMain(Int32 argc, String[] argv, IntPtr principalClassName, IntPtr delegateClassName)
   at UIKit.UIApplication.Main(String[] args, Type principalClass, Type delegateClass)
   at Program.<Main>$(String[] args) in /.../TestApp/Main.cs:line 6
marcb777 commented 1 year ago

I put together a small sample. It's a .Net7 iOS app and binding project. The binding project wraps a simple Xcode framework with a sync and async method that just outputs to console (also in the zip).

The sync method works fine from the C# testapp, but the async method crashes (when I don't have pinvoke wrappers enabled) with the error logged in iOS as: Microsoft.iOS: Unsupported type encoding: v@?16

I can't enabled pinvoke-wrappers due to the crash on launch in my previous comment.

There are bin logs for both the binding library build, as well as the TestApp, in the respective project folders. Sample.zip

rolfbjarne commented 1 year ago

@marcb777 I can reproduce the problem, but it works if you add this to the project as well:

<PropertyGroup>
    <Registrar>static</Registrar>
</PropertyGroup>
marcb777 commented 1 year ago

Ahh - thanks so much @rolfbjarne . That appears to have resolved the issue I was facing.

To be clear, is that needed on the bindings project, the app project, or both?

rolfbjarne commented 1 year ago

To be clear, is that needed on the bindings project, the app project, or both?

Just the app project.

rolfbjarne commented 11 months ago

Stackoverflow came to the rescue and found this:

https://github.com/llvm/llvm-project/blob/24a082878f7baec3651de56d54e5aa2b75a21b5f/clang/lib/AST/ASTContext.cpp#L8206-L8561

It looks like it's exactly what we need to implement this.