search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.44k stars 4.76k forks source link

WebSocket exception with LLVM and full AOT Enabled in .NET Android App #104599

Open kcrg opened 4 months ago

kcrg commented 4 months ago

Android framework version

net8.0-android, net9.0-android

Affected platform version

VS 17.11 Preview and 17.10 Stable, .NET 8.0.6 and .NET 9.0.0-preview.5.24306.8

Description

I am encountering an issue with WebSocket functionality in a .NET Android application when LLVM and full AOT is enabled. The application fails to establish WebSocket connections.

Configuration:

<PropertyGroup Condition="'$(Configuration)|$(TargetFramework)|$(Platform)'=='Release|net8.0-android|AnyCPU'">
  <EnableLLVM>True</EnableLLVM>
  <AndroidEnableProfiledAot>False</AndroidEnableProfiledAot>
  <RunAOTCompilation>True</RunAOTCompilation>
</PropertyGroup>

When running the application with the above configuration, WebSocket connections throws exception. The same code works without issues when LLVM is disabled.

Expected Behavior: The WebSocket connection should be established, messages should be sent and received without errors, and the connection should be properly closed.

Actual Behavior: WebSocket connection throws exception when LLVM is enabled with the above configuration.

Stack Trace:

Message: net_webstatus_ConnectFailure
StackTrace: at System.Net.WebSockets.WebSocketHandle.ConnectAsync(Uri , HttpMessageInvoker , CancellationToken , ClientWebSocketOptions )
   at System.Net.WebSockets.ClientWebSocket.ConnectAsyncCore(Uri , HttpMessageInvoker , CancellationToken )
   at WebsocketLLVMIssue.WebSocketClient.ConnectAsync(String uri)
   at WebsocketLLVMIssue.MainActivity.ConnectAsync()

Additional Notes:

Steps to Reproduce

  1. Create a new .NET Android project.
  2. Implement WebSocket client functionality.
  3. Add UI elements for connecting, sending messages, and disconnecting.
  4. Handle user interactions to manage WebSocket operations.
  5. Configure the project to use LLVM without AOT.
  6. Build and run the project on an Android device or emulator.
  7. Observe the behavior when trying to connect using WebSocket.

Or download this minimal project: WebsocketLLVMIssue.zip

Did you find any workaround?

nope

Relevant log output

logcat.txt

grendello commented 4 months ago

@kcrg Could you please collect more info for us by issuing the following commands from the VS Developer Prompt:

> adb shell setprop debug.mono.log default,assembly,mono_log_level=debug,mono_log_mask=all
> adb logcat -G 65M
> adb logcat -c
rem Start the crashing application here, wait for it to crash, then wait 5 more seconds and...
> adb logcat -d > logcat.txt

Then attach the resulting logcat.txt file to this issue, thanks!

kcrg commented 4 months ago

@grendello Done, I just added the logcat.txt to the issue. I had problem with adb shell debug.mono.log default,assembly,mono_log_level=debug,mono_log_mask=all it returns /system/bin/sh: debug.mono.log: inaccessible or not found

Issue occures only in Release. Enabling AOT and LLVM in Debug mode seems wrong.

grendello commented 4 months ago

@grendello Done, I just added the logcat.txt to the issue. I had problem with adb shell debug.mono.log default,assembly,mono_log_level=debug,mono_log_mask=all it returns /system/bin/sh: debug.mono.log: inaccessible or not found

Oops, my bad, sorry about that. The first command should be:

> adb shell setprop debug.mono.log default,assembly,mono_log_level=debug,mono_log_mask=all

And please attach logcat.txt directly here, to your next comment, thanks!

kcrg commented 4 months ago

@grendello Here it is: logcat.txt

grendello commented 4 months ago

@kcrg it looks like a runtime issue, I can't find anything that's specific to .NET for Android. @steveisok I don't have permissions to transfer this to dotnet/runtime, would you mind doing that and assigning somebody to take a look at this? Thanks!

dotnet-policy-service[bot] commented 4 months ago

Tagging subscribers to this area: @dotnet/ncl See info in area-owners.md if you want to be subscribed.

vitek-karas commented 4 months ago

Looking into the log, this seems network related:

07-08 15:32:19.972  1492  5110 D VpnManagerService: Exception: java.lang.NullPointerException: Attempt to invoke interface method 'int com.samsung.android.knox.zt.networktrust.filter.IKnoxNetworkFilterService.getKnoxNwFilterHttpProxyPort(int, java.lang.String)' on a null object reference
07-08 15:32:19.972  1492  5110 D VpnManagerService:     at com.android.server.VpnManagerService.getKnoxNwFilterHttpProxyPort(VpnManagerService.java:1945)
07-08 15:32:19.972  1492  5110 D VpnManagerService:     at android.net.IVpnManager$Stub.onTransact(IVpnManager.java:1075)
07-08 15:32:19.972  1492  5110 D VpnManagerService:     at android.os.Binder.execTransactInternal(Binder.java:1375)
07-08 15:32:19.972  1492  5110 D VpnManagerService:     at android.os.Binder.execTransact(Binder.java:1311)

@simonrozsival could you please take a quick look if this is something we've seen before or it's new...

grendello commented 4 months ago

@vitek-karas @simonrozsival that exception is a red herring, it comes from a different (system) process. The one that is relevant is the following:

07-08 15:32:19.814 15061 15061 D AndroidRuntime: Shutting down VM
--------- beginning of crash
07-08 15:32:19.814 15061 15061 E AndroidRuntime: FATAL EXCEPTION: main
07-08 15:32:19.814 15061 15061 E AndroidRuntime: Process: com.companyname.WebsocketLLVMIssue, PID: 15061
07-08 15:32:19.814 15061 15061 E AndroidRuntime: android.runtime.JavaProxyThrowable: [System.Net.WebSockets.WebSocketException]: net_webstatus_ConnectFailure
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Net.WebSockets.WebSocketHandle+<ConnectAsync>d__22.MoveNext(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Net.WebSockets.ClientWebSocket+<ConnectAsyncCore>d__21.MoveNext(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at WebsocketLLVMIssue.WebSocketClient+<ConnectAsync>d__4.MoveNext(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at WebsocketLLVMIssue.MainActivity+<ConnectAsync>d__5.MoveNext(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at WebsocketLLVMIssue.MainActivity+<<OnCreate>b__4_0>d.MoveNext(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Threading.Tasks.Task+<>c.<ThrowAsync>b__128_0(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at Android.App.SyncContext+<>c__DisplayClass2_0.<Post>b__0(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at Java.Lang.Thread+RunnableImplementor.Run(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at Java.Lang.IRunnableInvoker.n_Run(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at Android.Runtime.JNINativeWrapper.Wrap_JniMarshal_PP_V(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at mono.java.lang.RunnableImplementor.n_run(Native Method)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at mono.java.lang.RunnableImplementor.run(RunnableImplementor.java:31)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at android.os.Handler.handleCallback(Handler.java:958)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at android.os.Handler.dispatchMessage(Handler.java:99)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at android.os.Looper.loopOnce(Looper.java:230)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at android.os.Looper.loop(Looper.java:319)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at android.app.ActivityThread.main(ActivityThread.java:8919)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at java.lang.reflect.Method.invoke(Native Method)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:578)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1103)
...
07-08 15:32:19.818 15061 15061 I MonoDroid: Android.Runtime.JavaProxyThrowable: Exception_WasThrown, Android.Runtime.JavaProxyThrowable
07-08 15:32:19.818 15061 15061 I MonoDroid: 
07-08 15:32:19.818 15061 15061 I MonoDroid:   --- End of managed Android.Runtime.JavaProxyThrowable stack trace ---
07-08 15:32:19.818 15061 15061 I MonoDroid: android.runtime.JavaProxyThrowable: [System.Net.WebSockets.WebSocketException]: net_webstatus_ConnectFailure
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Net.WebSockets.WebSocketHandle+<ConnectAsync>d__22.MoveNext(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Net.WebSockets.ClientWebSocket+<ConnectAsyncCore>d__21.MoveNext(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at WebsocketLLVMIssue.WebSocketClient+<ConnectAsync>d__4.MoveNext(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at WebsocketLLVMIssue.MainActivity+<ConnectAsync>d__5.MoveNext(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at WebsocketLLVMIssue.MainActivity+<<OnCreate>b__4_0>d.MoveNext(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Threading.Tasks.Task+<>c.<ThrowAsync>b__128_0(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at Android.App.SyncContext+<>c__DisplayClass2_0.<Post>b__0(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at Java.Lang.Thread+RunnableImplementor.Run(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at Java.Lang.IRunnableInvoker.n_Run(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at Android.Runtime.JNINativeWrapper.Wrap_JniMarshal_PP_V(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at mono.java.lang.RunnableImplementor.n_run(Native Method)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at mono.java.lang.RunnableImplementor.run(RunnableImplementor.java:31)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at android.os.Handler.handleCallback(Handler.java:958)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at android.os.Handler.dispatchMessage(Handler.java:99)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at android.os.Looper.loopOnce(Looper.java:230)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at android.os.Looper.loop(Looper.java:319)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at android.app.ActivityThread.main(ActivityThread.java:8919)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at java.lang.reflect.Method.invoke(Native Method)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:578)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1103)
07-08 15:32:19.818 15061 15061 I MonoDroid: 
07-08 15:32:19.818 15061 15061 I MonoDroid:   --- End of managed Android.Runtime.JavaProxyThrowable stack trace ---
07-08 15:32:19.818 15061 15061 I MonoDroid: android.runtime.JavaProxyThrowable: [System.Net.WebSockets.WebSocketException]: net_webstatus_ConnectFailure
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Net.WebSockets.WebSocketHandle+<ConnectAsync>d__22.MoveNext(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Net.WebSockets.ClientWebSocket+<ConnectAsyncCore>d__21.MoveNext(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at WebsocketLLVMIssue.WebSocketClient+<ConnectAsync>d__4.MoveNext(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at WebsocketLLVMIssue.MainActivity+<ConnectAsync>d__5.MoveNext(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at WebsocketLLVMIssue.MainActivity+<<OnCreate>b__4_0>d.MoveNext(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Threading.Tasks.Task+<>c.<ThrowAsync>b__128_0(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at Android.App.SyncContext+<>c__DisplayClass2_0.<Post>b__0(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at Java.Lang.Thread+RunnableImplementor.Run(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at Java.Lang.IRunnableInvoker.n_Run(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at Android.Runtime.JNINativeWrapper.Wrap_JniMarshal_PP_V(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at mono.java.lang.RunnableImplementor.n_run(Native Method)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at mono.java.lang.RunnableImplementor.run(RunnableImplementor.java:31)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at android.os.Handler.handleCallback(Handler.java:958)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at android.os.Handler.dispatchMessage(Handler.java:99)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at android.os.Looper.loopOnce(Looper.java:230)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at android.os.Looper.loop(Looper.java:319)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at android.app.ActivityThread.main(ActivityThread.java:8919)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at java.lang.reflect.Method.invoke(Native Method)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:578)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1103)
07-08 15:32:19.818 15061 15061 I MonoDroid:

It is also related to networking, and the weird thing is that AOT+LLVM somehow makes the connection fail.

vitek-karas commented 4 months ago

Thanks a lot @grendello - learning :-)

simonrozsival commented 4 months ago

I can reproduce the issue locally, this is the exception stack trace I'm getting:

System.Net.WebSockets.WebSocketException (0x80004005): net_webstatus_ConnectFailure
 ---> System.Net.Http.HttpRequestException: Arg_InvalidOperationException (echo.websocket.org:443)
 ---> System.InvalidOperationException: Arg_InvalidOperationException
   at System.Threading.Tasks.Sources.ManualResetValueTaskSourceCore`1[[System.Boolean, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].GetStatus(Int16 )
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetStatus(Int16 )
   at System.Net.Sockets.Socket.ConnectAsync(EndPoint , CancellationToken )
   at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String , Int32 , HttpRequestMessage , Boolean , CancellationToken )
   Exception_EndOfInnerExceptionStack
   at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String , Int32 , HttpRequestMessage , Boolean , CancellationToken )
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage , Boolean , CancellationToken )
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage , Boolean , CancellationToken )
   at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem )
   at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.<WaitWithCancellationAsync>d__1[[System.Net.Http.HttpConnection, System.Net.Http, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].MoveNext()
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage , Boolean , Boolean , CancellationToken )
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage , Boolean , CancellationToken )
   at System.Net.WebSockets.WebSocketHandle.ConnectAsync(Uri , HttpMessageInvoker , CancellationToken , ClientWebSocketOptions )
   at System.Net.WebSockets.WebSocketHandle.ConnectAsync(Uri , HttpMessageInvoker , CancellationToken , ClientWebSocketOptions )
   at System.Net.WebSockets.ClientWebSocket.ConnectAsyncCore(Uri , HttpMessageInvoker , CancellationToken )
   at WebsocketLLVMIssue.WebSocketClient.ConnectAsync(String uri)
   at WebsocketLLVMIssue.MainActivity.ConnectAsync()

I get the same failure if I keep the same LLVM configuration, but I simply to connect a Socket:

try
{
    var socket = new Socket(SocketType.Stream, ProtocolType.Tcp) { NoDelay = true };
    await socket.ConnectAsync(IPEndPoint.Parse("1.1.1.1:443"), CancellationToken.None);
}
catch (Exception ex)
{
    Console.WriteLine(ex);
}

I get the following exception:

System.InvalidOperationException: Arg_InvalidOperationException
  at System.Threading.Tasks.Sources.ManualResetValueTaskSourceCore`1[[System.Boolean, System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].GetStatus(Int16 )
  at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetStatus(Int16 )
  at System.Net.Sockets.Socket.ConnectAsync(EndPoint , CancellationToken )
  at TestSocketsHttpHandlerLlvm.MainActivity.<>c.<<OnCreate>b__1_0>d.MoveNext()

This means that using SocketsHttpHandler with LLVM won't work.

@wfurt do you have an idea what could be going on?

wfurt commented 4 months ago

no, I don't know why AOT would make any difference. (assuming same code runs OK on Android without AOT) This seems to come from the ManualResetValueTaskSourceCore implementation.

        private void ValidateToken(short token)
        {
            if (token != _version)
            {
                ThrowHelper.ThrowInvalidOperationException();
            }
        }

perhaps you can look at it in debugger and check the token version(s). cc @stephentoub for any more recommendations.

matouskozak commented 3 months ago

Investigation update

The failures does reproduce in dotnet new android and dotnet new maui (net9.0-android) apps with code from https://github.com/dotnet/runtime/issues/104599#issuecomment-2225787986 with AOT-llvm:

  <EnableLLVM>True</EnableLLVM>
  <AndroidEnableProfiledAot>False</AndroidEnableProfiledAot>
  <RunAOTCompilation>True</RunAOTCompilation>

However, I was not able to reproduce the same issue on iOS (which uses AOT-llvm as well) or with a Mono sample app on arm64 desktop in AOT-llvm configuration. What's more, the Android sample app in dotnet/runtime also doesn't reproduce the issue (setting Makefilevariables USE_LLVM=true and AOT=true).

I'm thinking that there might be a conflict between dotnet/runtime and dotnet/android LLVMs which is causing some unexpected behavior. @jonathanpeppers do you know who would be the best person to discuss this issue with?

jonathanpeppers commented 3 months ago

@grendello or I might be able to help, depending what the problem is. Do you know if there is some AOT compiler flag/option we are missing that iOS has?

Offhand, -p:EnableLLVM=true will require the Android NDK, while other builds can succeed without the NDK used at all.

grendello commented 3 months ago

One thing that might be wrong is the newer LLVM linkers tend to rewrite code at link time and that breaks Mono's AOT since it expects to find the exact same code at run time as it generated at compile time. To prevent lld from doing this, we pass the --no-relax flag to it. Perhaps this flag is somehow missing when LLVM AOT is enabled? Even if it's there, it might be a good idea to compare disassembled source of the involved AOT shared libraries around the WebSocket code, when built with and without LLVM?

matouskozak commented 3 months ago

One thing that might be wrong is the newer LLVM linkers tend to rewrite code at link time and that breaks Mono's AOT since it expects to find the exact same code at run time as it generated at compile time. To prevent lld from doing this, we pass the --no-relax flag to it. Perhaps this flag is somehow missing when LLVM AOT is enabled? Even if it's there, it might be a good idea to compare disassembled source of the involved AOT shared libraries around the WebSocket code, when built with and without LLVM?

I checked the binlogs and the --no-relax flag seems to be applied: [System.Net.Sockets.dll] Exec (with response file contents expanded) in .../WebSocketLLVMExceptionMAUI: MONO_PATH=.../WebSocketLLVMExceptionMAUI/obj/Release/net9.0-android/android-x86/linked: MONO_ENV_OPTIONS= /usr/local/share/dotnet/packs/Microsoft.NETCore.App.Runtime.AOT.osx-arm64.Cross.android-x86/9.0.0-preview.4.24266.19/Sdk/../tools/mono-aot-cross --debug --llvm "--aot=asmwriter,temp-path=obj/Release/net9.0-android/android-x86/aot/x86/System.Net.Sockets,nodebug,llvm-path=/usr/local/share/dotnet/packs/Microsoft.NETCore.App.Runtime.AOT.osx-arm64.Cross.android-x86/9.0.0-preview.4.24266.19/Sdk/../tools,mtriple=i686-linux-android,tool-prefix=/usr/local/share/dotnet/packs/Microsoft.Android.Sdk.Darwin/34.99.0-preview.4.272/tools/Darwin/binutils/bin/i686-linux-android-,outfile=obj/Release/net9.0-android/android-x86/aot/System.Net.Sockets.dll.so,llvm-outfile=obj/Release/net9.0-android/android-x86/aot/System.Net.Sockets.dll-llvm.o,temp-path=obj/Release/net9.0-android/android-x86/System.Net.Sockets.dll.tmp,ld-name=ld,ld-flags=\"/usr/local/share/dotnet/packs/Microsoft.Android.Sdk.Darwin/34.99.0-preview.4.272/tools/libstubs/android-x86/libc.so\";\"/usr/local/share/dotnet/packs/Microsoft.Android.Sdk.Darwin/34.99.0-preview.4.272/tools/libstubs/android-x86/libm.so\" --no-relax -s" "obj/Release/net9.0-android/android-x86/linked/System.Net.Sockets.dll" (here is the full binlog).

@grendello or I might be able to help, depending what the problem is. Do you know if there is some AOT compiler flag/option we are missing that iOS has?

Offhand, -p:EnableLLVM=true will require the Android NDK, while other builds can succeed without the NDK used at all.

What I've noticed that is that on iOS, we use by default MONO_AOT_MODE_FULL, whereas on Android we set only MONO_AOT_MODE_NORMAL (allows JIT). Not sure if that could play a role as well.

I'll try to get the disassembled source of System.Net.Sockets dll and I'll report the findings here.

@BrzVlad any thoughts on how to investigate this?

DigitApps2 commented 3 months ago

We have the same issue using latest Azure.Storage.Blobs nuget for the azure file upload functions

alexyakunin commented 2 months ago

Experiencing the same issue - we noticed it a while ago (maybe 6mo or so), didn't have a chance to report.

The exception we get is identical to this one: https://github.com/dotnet/runtime/issues/104599#issuecomment-2225787986

alexyakunin commented 2 months ago

Just captured our own exception: image

alexyakunin commented 2 months ago

Just checked the sample from .zip on .NET 9 Preview 7: it works.

That was the only modification (+ install whatever is required):

    <TargetFramework>net9.0-android</TargetFramework>
kcrg commented 2 months ago

@alexyakunin By changing only target framework to .NET 9, you disable the below PropertyGroup with Condition="'$(Configuration)|$(TargetFramework)|$(Platform)'=='Release|net8.0-android|AnyCPU'"

If u want to test if it works on .NET 9, update Condition in this PropertyGroup to .NET 9.

Here's how csproj looks like on .NET 9:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net9.0-android</TargetFramework>
    <SupportedOSPlatformVersion>21</SupportedOSPlatformVersion>
    <OutputType>Exe</OutputType>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
    <ApplicationId>com.companyname.WebsocketLLVMIssue</ApplicationId>
    <ApplicationVersion>1</ApplicationVersion>
    <ApplicationDisplayVersion>1.0</ApplicationDisplayVersion>
  </PropertyGroup>

  <PropertyGroup Condition="'$(Configuration)|$(TargetFramework)|$(Platform)'=='Release|net9.0-android|AnyCPU'">
    <EnableLLVM>True</EnableLLVM>
    <AndroidEnableProfiledAot>False</AndroidEnableProfiledAot>
    <RunAOTCompilation>True</RunAOTCompilation>
  </PropertyGroup>
</Project>

The issue still persists in .NET 9 rc1.

alexyakunin commented 2 months ago

@kcrg Oh, you're right - I forgot to change the framework identifier in Condition part.

Which means we also really need this fix.

guygodin commented 2 months ago

Also experiencing this issue when attempting to bring a Xamarin.Android app to .NET 8.0/9.0 while using regular TCP sockets. Same exception as described here: https://github.com/dotnet/runtime/issues/104599#issuecomment-2225787986

alexyakunin commented 1 month ago

Hi guys, are there any updates on that? In particular, will the issue with WebSockets and sockets be addressed @ .NET 9 release?

matouskozak commented 1 month ago

Hi guys, are there any updates on that? In particular, will the issue with WebSockets and sockets be addressed @ .NET 9 release?

The issue was moved to .NET 10 so the fix likely won't be part of the .NET 9 release but will be backported to .NET 9 servicing when ready.

We don't have any update yet as we are prioritizing .NET 9 issues before the release. In the meantime, could you try setting disabling LLVM to work around this issue (<EnableLLVM>False</EnableLLVM>)?

alexyakunin commented 1 month ago

In the meantime, could you try setting disabling LLVM to work around this issue (<EnableLLVM>False</EnableLLVM>)?

It works in this case - both in our app and in the demo app above.

matouskozak commented 1 month ago

In the meantime, could you try setting disabling LLVM to work around this issue (<EnableLLVM>False</EnableLLVM>)?

It works in this case - both in our app and in the demo app above.

Thank you for the update. Could you please share the reasons behind enabling LLVM on Android? I don't think it is enabled by default.

kcrg commented 1 month ago

Thank you for the update. Could you please share the reasons behind enabling LLVM on Android? I don't think it is enabled by default.

https://learn.microsoft.com/en-us/dotnet/maui/whats-new/dotnet-9?view=net-maui-8.0#llvm-marshalled-methods

Low-level Virtual Machine (LLVM) marshalled methods are now enabled by default in .NET for Android 9 in non-Blazor apps.

jonathanpeppers commented 1 month ago

@kcrg $(AndroidEnableMarshalMethods) is not the same feature as $(EnableLLVM). I'll look into updating this doc, it also seems like the MSBuild properties they mention here are not spelled correctly?!?

$(EnableLLVM) is a Mono AOT compiler feature that has been around since Xamarin, but it has some issues that prevent us from making it the default.

kcrg commented 1 month ago

@jonathanpeppers so what is the difference between <AndroidEnableLLVM>True</AndroidEnableLLVM> and <EnableLLVM>True</EnableLLVM>?

This is confusing to me.

Also

... by default in .NET for Android 9 in non-Blazor apps.

First thought was that it applies to Android 9, second thought was that I'm reading the .NET for Android release notes, so it applies to .NET 9 itself. I know there was a name change Xamarin.Android -> .NET for Android, I think a better option is to put the version number right after ".NET" -> .NET 9 for Android

jonathanpeppers commented 1 month ago

@jonathanpeppers so what is the difference between <AndroidEnableLLVM>True</AndroidEnableLLVM> and <EnableLLVM>True</EnableLLVM>?

$(AndroidEnableLLVM) looks to be an AI hallucination, it does not exist:

alexyakunin commented 1 month ago

Okey, a quick update: you can workaround this issue by adding this piece to .csproj:

  <!--
  LLVM fails in runtime if it processes System.Net.Sockets.dll.
  See https://github.com/dotnet/runtime/issues/104599
  -->
  <Target Name="_FixAndroidAotInputs" DependsOnTargets="_AndroidAotInputs" BeforeTargets="_AndroidAotCompilation">
    <ItemGroup Condition="$(EnableLLVM)">
      <_AndroidAotInputs Remove="$(IntermediateLinkDir)**\System.Net.Sockets.dll" />
    </ItemGroup>
  </Target>

More sophisticated fix idea is to generate AOT build of System.Net.Sockets.dll w/o LLVM. I don't have time to work on it now, + I suspect it won't change much in terms of perf. So if someone else comes up w/ the modifications needed, please share.

CC @kcrg

matouskozak commented 1 month ago

Possibly the same root cause as https://github.com/dotnet/runtime/issues/104167