App crash due to NullReferenceException in NSUrlSessionHandlerDelegate.DidCompleteWithError

adams-family commented 3 years ago

IMPORTANT:

I think this problem will be somewhat related to: https://github.com/xamarin/xamarin-macios/issues/9132

Steps to Reproduce

I happens on iOS, latest versions (latest iOS, latest Xamarin) very often in production (tens of crashes per day on a 2k user basis), randomly. It's hard to reproduce it in the debugger.

protected HttpClient client = new HttpClient();

void Test()
{
  try
  {
     var stringContent = new StringContent(json, UnicodeEncoding.UTF8, "application/json");
     stringContent.Headers.Add("sample", "value");
     var response = await client.PostAsync("https://....", stringContent);
  }
  catch (Exception e)
  {
     // it never hits this exception handler
  }
}

Expected Behavior

Thanks to the try/catch block this code should never cause an app crash.

Actual Behavior

The app crashes. Here is an AppCenter stacktrace:

NSUrlSessionHandler+NSUrlSessionHandlerDelegate.DidCompleteWithError (Foundation.NSUrlSession session, Foundation.NSUrlSessionTask task, Foundation.NSError error)

Environment

Microsoft Visual Studio Community 2019
Version 16.8.4
VisualStudio.16.Release/16.8.4+30907.101
Microsoft .NET Framework
Version 4.8.04084

Installed Version: Community

Visual C++ 2019   00435-60000-00000-AA591
Microsoft Visual C++ 2019

ASP.NET and Web Tools 2019   16.8.557.25636
ASP.NET and Web Tools 2019

Azure App Service Tools v3.0.0   16.8.557.25636
Azure App Service Tools v3.0.0

C# Tools   3.8.0-5.20604.10+9ed4b774d20940880de8df1ca8b07508aa01c8cd
C# components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.

Common Azure Tools   1.10
Provides common services for use by Azure Mobile Services and Microsoft Azure Tools.

Extensibility Message Bus   1.2.6 (master@34d6af2)
Provides common messaging-based MEF services for loosely coupled Visual Studio extension components communication and integration.

IncrediBuild Build Acceleration   1.5.0.13
IncrediBuild effectively reduces compilation and development times by up to 90%.

IntelliCode Extension   1.0
IntelliCode Visual Studio Extension Detailed Info

Microsoft JVM Debugger   1.0
Provides support for connecting the Visual Studio debugger to JDWP compatible Java Virtual Machines

Microsoft MI-Based Debugger   1.0
Provides support for connecting Visual Studio to MI compatible debuggers

Microsoft Visual C++ Wizards   1.0
Microsoft Visual C++ Wizards

Microsoft Visual Studio VC Package   1.0
Microsoft Visual Studio VC Package

Mono Debugging for Visual Studio   16.8.43 (00471f8)
Support for debugging Mono processes with Visual Studio.

NuGet Package Manager   5.8.1
NuGet Package Manager in Visual Studio. For more information about NuGet, visit https://docs.nuget.org/

ProjectServicesPackage Extension   1.0
ProjectServicesPackage Visual Studio Extension Detailed Info

Qt Visual Studio Tools   2.7.1.18
The Qt Visual Studio Tools allow developers to use the standard development environment without having to worry about any Qt-related build steps or tools.

Syncfusion ASP.NET Core Project Templates   18.1.0.42
Provides project templates for creating an ASP.NET Core Web Application using Syncfusion ASP.NET Core components.

Syncfusion ASP.NET Core Project Templates   16.1.0.32
Provides project templates for creating an ASP.NET Core Web Application using Syncfusion ASP.NET Core components.

Syncfusion ASP.NET MVC Project Templates   17.1.0.32
Provides project templates for creating ASP.NET MVC Web Applications using Syncfusion components.

Syncfusion Xamarin Project Templates   13.4.0.53
A multi-project template for building iOS, Android, and Windows apps using Syncfusion Xamarin components.

Syncfusion Xamarin Toolbox   16.2.0.19
Syncfusion Essential Studio for Xamarin Toolbox Visual Studio extension.

SyncfusionMenu Extension   1.0
SyncfusionMenu Visual Studio Extension Detailed Info

Test Adapter for Boost.Test   1.0
Enables Visual Studio's testing tools with unit tests written for Boost.Test.  The use terms and Third Party Notices are available in the extension installation directory.

Test Adapter for Google Test   1.0
Enables Visual Studio's testing tools with unit tests written for Google Test.  The use terms and Third Party Notices are available in the extension installation directory.

Visual Basic Tools   3.8.0-5.20604.10+9ed4b774d20940880de8df1ca8b07508aa01c8cd
Visual Basic components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.

Visual F# Tools   16.8.0-beta.20507.4+da6be68280c89131cdba2045525b80890401defd
Microsoft Visual F# Tools

Visual Studio Code Debug Adapter Host Package   1.0
Interop layer for hosting Visual Studio Code debug adapters in Visual Studio

Visual Studio Tools for CMake   1.0
Visual Studio Tools for CMake

VisualStudio.DeviceLog   1.0
Information about my package

VisualStudio.Foo   1.0
Information about my package

VisualStudio.Mac   1.0
Mac Extension for Visual Studio

Xamarin   16.8.000.261 (d16-8@bb99248)
Visual Studio extension to enable development for Xamarin.iOS and Xamarin.Android.

Xamarin Designer   16.8.0.507 (remotes/origin/d16-8@e87b24884)
Visual Studio extension to enable Xamarin Designer tools in Visual Studio.

Xamarin Templates   16.8.112 (86385a3)
Templates for building iOS, Android, and Windows apps with Xamarin and Xamarin.Forms.

Xamarin.Android SDK   11.1.0.26 (d16-8/a36ce73)
Xamarin.Android Reference Assemblies and MSBuild support.
    Mono: 5e9cb6d
    Java.Interop: xamarin/java.interop/d16-8@79d9533
    ProGuard: Guardsquare/proguard/proguard6.2.2@ebe9000
    SQLite: xamarin/sqlite/3.32.1@1a3276b
    Xamarin.Android Tools: xamarin/xamarin-android-tools/d16-8@2fb1cbc

Xamarin.iOS and Xamarin.Mac SDK   14.8.0.3 (c51fabee8)
Xamarin.iOS and Xamarin.Mac Reference Assemblies and MSBuild support.

Example Project (If Possible)

I don't think it's possible to create one because I'd need to roll it out to at least a thousand users.

spouliot commented 3 years ago

I think this problem will be somewhat related to: #9132

c.c. @mandel-macaque

I don't think it's possible to create one because I'd need to roll it out to at least a thousand users.

Can you provide us with a complete, symbolicated crash report ?

Sadly the stack trace above is not telling us much about what's happening.

Thanks!

mandel-macaque commented 3 years ago

@spouliot @adams-family looking at the Xamarin.iOS version:

Xamarin.iOS and Xamarin.Mac SDK 14.8.0.3 (c51fabee8)

That points to commit: https://github.com/xamarin/xamarin-macios/commit/c51fabee8 that commits is earlier than the fix provided in https://github.com/xamarin/xamarin-macios/pull/10230

If you look at the source code of the commit of the version you are using https://github.com/xamarin/xamarin-macios/blob/c51fabee8aa096e83f7a781aecba68113613b71d/src/Foundation/NSUrlSessionHandler.cs#L543 you can see that the fix I provided is not there. It is the same bug, but you are in a much older version which means that you do not have the fix.

The current stable release is https://download.visualstudio.microsoft.com/download/pr/414d1675-ca0b-4b5b-8a4e-26b693883581/5e50228d1498caf85151b6d6cfec6fd9/xamarin.ios-14.20.0.1.pkg

Please, make sure that you are running the latests stable and then we can re-visit the issue.

(Edit: updated pkg to d16-10 which also has the fix)

adams-family commented 3 years ago

Hi @mandel-macaque

I double-checked and the machine used to build the iOS version before releasing to the AppStore was this one: Xamarin.iOS 14.14.2.5.

=== Visual Studio Community 2019 for Mac ===

Version 8.9.1 (build 34)
Installation UUID: 7d21187b-92bc-43a9-ab04-339062ddf7e1
    GTK+ 2.24.23 (Raleigh theme)
    Xamarin.Mac 6.18.0.23 (d16-6 / 088c73638)

    Package version: 612000122

=== Mono Framework MDK ===

Runtime:
    Mono 6.12.0.122 (2020-02/c621c35ffa0) (64-bit)
    Package version: 612000122

=== Roslyn (Language Service) ===

3.9.0-6.21152.10+c10f884b30737542ddd84ca889a4aad9281ce210

=== NuGet ===

Version: 5.8.0.6860

=== .NET Core SDK ===

SDK: /usr/local/share/dotnet/sdk/5.0.201/Sdks
SDK Versions:
    5.0.201
    3.1.407
MSBuild SDKs: /Applications/Visual Studio.app/Contents/Resources/lib/monodevelop/bin/MSBuild/Current/bin/Sdks

=== .NET Core Runtime ===

Runtime: /usr/local/share/dotnet/dotnet
Runtime Versions:
    5.0.4
    3.1.13

=== .NET Core 3.1 SDK ===

SDK: 3.1.407

=== Xamarin.Profiler ===

Version: 1.6.15.68
Location: /Applications/Xamarin Profiler.app/Contents/MacOS/Xamarin Profiler

=== Updater ===

Version: 11

=== Apple Developer Tools ===

Xcode 12.5 (18205)
Build 12E262

=== Xamarin.Mac ===

Xamarin.Mac not installed. Can't find /Library/Frameworks/Xamarin.Mac.framework/Versions/Current/Version.

=== Xamarin.iOS ===

Version: 14.14.2.5 (Visual Studio Community)
Hash: 3836759d4
Branch: d16-9
Build date: 2021-02-10 17:56:44-0500

=== Xamarin Designer ===

Version: 16.9.0.316
Hash: bd2705417
Branch: remotes/origin/d16-9
Build date: 2021-02-24 00:16:08 UTC

=== Xamarin.Android ===

Version: 11.2.0.21 (Visual Studio Community)
Commit: xamarin-android/d16-9/93eab59
Android SDK: /Users/adam/Library/Developer/Xamarin/android-sdk-macosx
    Supported Android versions:
        None installed

SDK Tools Version: 26.1.1
SDK Platform Tools Version: 30.0.4
SDK Build Tools Version: 30.0.2

Build Information: 
Mono: 5e9cb6d
Java.Interop: xamarin/java.interop/d16-9@d6d86b2
ProGuard: Guardsquare/proguard/v7.0.1@912d149
SQLite: xamarin/sqlite/3.34.1@daff8f4
Xamarin.Android Tools: xamarin/xamarin-android-tools/d16-9@9d8924d

=== Microsoft OpenJDK for Mobile ===

Java SDK: /Users/adam/Library/Developer/Xamarin/jdk/microsoft_dist_openjdk_1.8.0.25
1.8.0-25
Android Designer EPL code available here:
https://github.com/xamarin/AndroidDesigner.EPL

=== Android SDK Manager ===

Version: 16.9.0.22
Hash: a391de2
Branch: remotes/origin/d16-9
Build date: 2021-02-18 03:14:56 UTC

=== Android Device Manager ===

Version: 16.9.0.17
Hash: fc2b3db
Branch: remotes/origin/d16-9
Build date: 2021-02-18 03:15:18 UTC

=== Build Information ===

Release ID: 809010034
Git revision: 33bce2b7f3d45d05865cd3282f1c2cdc3a112aa0
Build date: 2021-03-04 11:21:38-05
Build branch: release-8.9
Xamarin extensions: 33bce2b7f3d45d05865cd3282f1c2cdc3a112aa0

=== Operating System ===

Mac OS X 10.16.0
Darwin 20.3.0 Darwin Kernel Version 20.3.0
    Thu Jan 21 00:07:06 PST 2021
    root:xnu-7195.81.3~1/RELEASE_X86_64 x86_64

adams-family commented 3 years ago

@spouliot - I have exported a crashreport from MS AppCenter, symbols were uploaded earlier therefore I hope it's useful to you:

Incident Identifier: d4c93912-92f7-4860-8988-***********
CrashReporter Key:   AB6B1072-8B29-4B32-8D24-***********
Hardware Model:      iPhone12,1
Process:         application.iOS [743]
Path:            /private/var/containers/Bundle/Application/B99CA9E0-70E0-4203-9EA3-7EB8B6B892B1/application/application.iOS
Identifier:      *****
Version:         1.1.6 (184)
Code Type:       arm64
Parent Process:   [1]

Date/Time:       2021-06-03T10:08:06.999Z
Launch Time:     2021-06-01T03:28:08Z
OS Version:      iPhone OS 14.4.2 (18D70)
Report Version:  104

Exception Type:  SIGABRT
Exception Codes: #0 at 0x1b4f51414
Crashed Thread:  20

Application Specific Information:
*** Terminating app due to uncaught exception 'SIGABRT', reason: 'Object reference not set to an instance of an object'

Xamarin Exception Stack:
System.NullReferenceException: Object reference not set to an instance of an object
  at System.Net.Http.NSUrlSessionHandler+NSUrlSessionHandlerDelegate.DidCompleteWithError (Foundation.NSUrlSession session, Foundation.NSUrlSessionTask task, Foundation.NSError error) <0x105cc29f0 + 0x00074> in <ad1be307855442e4b2cb88e33f94f908#8bd8601acf85cfa73490ae975776a7b0>:0

Thread 20 Crashed:
0   libsystem_kernel.dylib               0x00000001b4f51414 __pthread_kill + 8
1   libsystem_c.dylib                    0x000000019042fb74 abort + 100
2   application.iOS                      0x000000010671f9f8 0x104d98000 + 26769912
3   application.iOS                      0x00000001065e494c 0x104d98000 + 25479500
4   application.iOS                      0x0000000106579718 0x104d98000 + 25040664
5   application.iOS                      0x0000000106578100 0x104d98000 + 25035008
6   application.iOS                      0x000000010656e404 0x104d98000 + 24994820

Thread 0:
0   application.iOS                      0x00000001051b7fe0 0x104d98000 + 4325344
1   application.iOS                      0x00000001050353d4 0x104d98000 + 2741204
2   application.iOS                      0x0000000104edd180 0x104d98000 + 1331584
3   application.iOS                      0x0000000104edaa74 0x104d98000 + 1321588
4   application.iOS                      0x0000000104f8fd24 0x104d98000 + 2063652
5   application.iOS                      0x0000000104eb1fd0 0x104d98000 + 1155024
6   application.iOS                      0x0000000104eb1e70 0x104d98000 + 1154672
7   application.iOS                      0x0000000104f85b5c 0x104d98000 + 2022236
8   application.iOS                      0x0000000104f85be4 0x104d98000 + 2022372
9   application.iOS                      0x000000010501388c 0x104d98000 + 2603148
10  application.iOS                      0x0000000105013454 0x104d98000 + 2602068
11  application.iOS                      0x0000000105012f8c 0x104d98000 + 2600844
12  application.iOS                      0x0000000105013afc 0x104d98000 + 2603772
13  application.iOS                      0x0000000105013bd4 0x104d98000 + 2603988
14  application.iOS                      0x0000000104ed65b4 0x104d98000 + 1303988
15  application.iOS                      0x0000000104ebdfb4 0x104d98000 + 1204148
16  application.iOS                      0x0000000104ebe34c 0x104d98000 + 1205068
17  application.iOS                      0x0000000104ebe16c 0x104d98000 + 1204588
18  application.iOS                      0x00000001060a3b98 0x104d98000 + 19970968
19  application.iOS                      0x0000000104f6e7fc 0x104d98000 + 1927164
20  application.iOS                      0x0000000104ef670c 0x104d98000 + 1435404
21  application.iOS                      0x0000000104ef650c 0x104d98000 + 1434892
22  application.iOS                      0x0000000104f6e6b8 0x104d98000 + 1926840
23  application.iOS                      0x0000000104f12e98 0x104d98000 + 1552024
24  application.iOS                      0x0000000105d3d78c 0x104d98000 + 16406412
25  application.iOS                      0x0000000105178a00 0x104d98000 + 4065792
26  application.iOS                      0x000000010658b33c 0x104d98000 + 25113404
27  application.iOS                      0x0000000106646a60 0x104d98000 + 25881184
28  application.iOS                      0x000000010664a2c4 0x104d98000 + 25895620
29  application.iOS                      0x0000000104dd6020 0x104d98000 + 253984
30  application.iOS                      0x0000000104dd8040 0x104d98000 + 262208
31  Foundation                           0x0000000188397b90 __NSThreadPerformPerform + 184
32  CoreFoundation                       0x0000000186f7d76c __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 24
33  CoreFoundation                       0x0000000186f7d668 __CFRunLoopDoSource0 + 204
34  CoreFoundation                       0x0000000186f7c960 __CFRunLoopDoSources0 + 264
35  CoreFoundation                       0x0000000186f76a8c __CFRunLoopRun + 820
36  CoreFoundation                       0x0000000186f7621c CFRunLoopRunSpecific + 596
37  GraphicsServices                     0x000000019eb40784 GSEventRunModal + 160
38  UIKitCore                            0x00000001899b6ee8 -[UIApplication _run] + 1068
39  UIKitCore                            0x00000001899bc75c UIApplicationMain + 164
40  application.iOS                      0x0000000105d950d8 0x104d98000 + 16765144
41  application.iOS                      0x0000000105cf1ee0 0x104d98000 + 16096992
42  application.iOS                      0x0000000105cf1e64 0x104d98000 + 16096868
43  application.iOS                      0x0000000104dff744 0x104d98000 + 423748
44  application.iOS                      0x0000000105178a00 0x104d98000 + 4065792
45  application.iOS                      0x000000010658b33c 0x104d98000 + 25113404
46  application.iOS                      0x0000000106646a60 0x104d98000 + 25881184
47  application.iOS                      0x000000010664ccb8 0x104d98000 + 25906360
48  application.iOS                      0x00000001065698d0 0x104d98000 + 24975568
49  application.iOS                      0x0000000106727488 0x104d98000 + 26801288
50  application.iOS                      0x0000000104dff650 0x104d98000 + 423504
51  libdyld.dylib                        0x0000000186c366b0 start + 0

Thread 1:
0   libsystem_kernel.dylib               0x00000001b4f511ac __psynch_cvwait + 8
1   application.iOS                      0x00000001066ea3d0 0x104d98000 + 26551248
2   libsystem_pthread.dylib              0x00000001d2aabcb0 _pthread_start + 316
3   libsystem_pthread.dylib              0x00000001d2ab4778 thread_start + 4

Thread 2:
0   libsystem_kernel.dylib               0x00000001b4f2d30c semaphore_wait_trap + 8
1   application.iOS                      0x0000000106694a4c 0x104d98000 + 26200652
2   application.iOS                      0x00000001066948d0 0x104d98000 + 26200272
3   libsystem_pthread.dylib              0x00000001d2aabcb0 _pthread_start + 316
4   libsystem_pthread.dylib              0x00000001d2ab4778 thread_start + 4

Thread 3:
0   libsystem_kernel.dylib               0x00000001b4f2d2d0 mach_msg_trap + 8
1   CoreFoundation                       0x0000000186f7cc30 __CFRunLoopServiceMachPort + 376
2   CoreFoundation                       0x0000000186f76c14 __CFRunLoopRun + 1212
3   CoreFoundation                       0x0000000186f7621c CFRunLoopRunSpecific + 596
4   Foundation                           0x0000000188225df0 -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 228
5   Foundation                           0x0000000188225cbc -[NSRunLoop(NSRunLoop) runUntilDate:] + 88
6   UIKitCore                            0x0000000189a6ac48 -[UIEventFetcher threadMain] + 512
7   Foundation                           0x0000000188397a34 __NSThread__start__ + 860
8   libsystem_pthread.dylib              0x00000001d2aabcb0 _pthread_start + 316
9   libsystem_pthread.dylib              0x00000001d2ab4778 thread_start + 4

Thread 4:
0   libsystem_kernel.dylib               0x00000001b4f511ac __psynch_cvwait + 8
1   application.iOS                      0x00000001066f9b60 0x104d98000 + 26614624
2   application.iOS                      0x00000001066a00c4 0x104d98000 + 26247364
3   application.iOS                      0x000000010669ff9c 0x104d98000 + 26247068
4   application.iOS                      0x00000001066a01c0 0x104d98000 + 26247616
5   application.iOS                      0x000000010668ef98 0x104d98000 + 26177432
6   application.iOS                      0x0000000106609334 0x104d98000 + 25629492
7   application.iOS                      0x0000000104eff558 0x104d98000 + 1471832
8   application.iOS                      0x0000000104eff0c4 0x104d98000 + 1470660
9   application.iOS                      0x0000000104efebcc 0x104d98000 + 1469388
10  application.iOS                      0x0000000104efeb7c 0x104d98000 + 1469308
11  application.iOS                      0x0000000104efe99c 0x104d98000 + 1468828
12  application.iOS                      0x0000000104efeae8 0x104d98000 + 1469160
13  application.iOS                      0x0000000104f01e14 0x104d98000 + 1482260
14  application.iOS                      0x0000000104ef8d2c 0x104d98000 + 1445164
15  application.iOS                      0x0000000104ef670c 0x104d98000 + 1435404
16  application.iOS                      0x0000000104ef650c 0x104d98000 + 1434892
17  application.iOS                      0x0000000104ef6490 0x104d98000 + 1434768
18  application.iOS                      0x0000000104ef8e94 0x104d98000 + 1445524
19  application.iOS                      0x0000000105178a00 0x104d98000 + 4065792
20  application.iOS                      0x000000010658b33c 0x104d98000 + 25113404
21  application.iOS                      0x0000000106646a60 0x104d98000 + 25881184
22  application.iOS                      0x000000010664c34c 0x104d98000 + 25903948
23  application.iOS                      0x0000000106694b44 0x104d98000 + 26200900
24  application.iOS                      0x00000001066948d0 0x104d98000 + 26200272
25  libsystem_pthread.dylib              0x00000001d2aabcb0 _pthread_start + 316
26  libsystem_pthread.dylib              0x00000001d2ab4778 thread_start + 4

Thread 5:
0   libsystem_kernel.dylib               0x00000001b4f2d2d0 mach_msg_trap + 8
1   CoreFoundation                       0x0000000186f7cc30 __CFRunLoopServiceMachPort + 376
2   CoreFoundation                       0x0000000186f76c14 __CFRunLoopRun + 1212
3   CoreFoundation                       0x0000000186f7621c CFRunLoopRunSpecific + 596
4   AudioSession                         0x000000018eabf0c4 GenericRunLoopThread::Entry(void*) + 160
5   AudioSession                         0x000000018eac125c CAPThread::Entry(CAPThread*) + 88
6   libsystem_pthread.dylib              0x00000001d2aabcb0 _pthread_start + 316
7   libsystem_pthread.dylib              0x00000001d2ab4778 thread_start + 4

Thread 6:
0   libsystem_kernel.dylib               0x00000001b4f2d2d0 mach_msg_trap + 8
1   CoreFoundation                       0x0000000186f7cc30 __CFRunLoopServiceMachPort + 376
2   CoreFoundation                       0x0000000186f76c14 __CFRunLoopRun + 1212
3   CoreFoundation                       0x0000000186f7621c CFRunLoopRunSpecific + 596
4   CFNetwork                            0x000000018784c1c0 _CFURLStorageSessionCopyCache + 64448
5   Foundation                           0x0000000188397a34 __NSThread__start__ + 860
6   libsystem_pthread.dylib              0x00000001d2aabcb0 _pthread_start + 316
7   libsystem_pthread.dylib              0x00000001d2ab4778 thread_start + 4

Thread 7:
0   libsystem_kernel.dylib               0x00000001b4f511ac __psynch_cvwait + 8
1   application.iOS                      0x00000001066f9b88 0x104d98000 + 26614664
2   application.iOS                      0x00000001066a00c4 0x104d98000 + 26247364
3   application.iOS                      0x000000010669ff9c 0x104d98000 + 26247068
4   application.iOS                      0x0000000106638db4 0x104d98000 + 25824692
5   application.iOS                      0x0000000106607918 0x104d98000 + 25622808
6   application.iOS                      0x0000000104ef8418 0x104d98000 + 1442840
7   application.iOS                      0x0000000104ef84d0 0x104d98000 + 1443024
8   application.iOS                      0x0000000104ef7fbc 0x104d98000 + 1441724
9   application.iOS                      0x0000000104ef8010 0x104d98000 + 1441808
10  application.iOS                      0x0000000104ef2ff8 0x104d98000 + 1421304
11  application.iOS                      0x0000000104ef2d5c 0x104d98000 + 1420636
12  application.iOS                      0x0000000105782750 0x104d98000 + 10397520
13  application.iOS                      0x0000000105782548 0x104d98000 + 10397000
14  application.iOS                      0x000000010578247c 0x104d98000 + 10396796
15  application.iOS                      0x0000000105f43734 0x104d98000 + 18528052
16  application.iOS                      0x0000000104ef8d2c 0x104d98000 + 1445164
17  application.iOS                      0x0000000104ef670c 0x104d98000 + 1435404
18  application.iOS                      0x0000000104ef650c 0x104d98000 + 1434892
19  application.iOS                      0x0000000104ef6490 0x104d98000 + 1434768
20  application.iOS                      0x0000000104ef8e94 0x104d98000 + 1445524
21  application.iOS                      0x0000000105178a00 0x104d98000 + 4065792
22  application.iOS                      0x000000010658b33c 0x104d98000 + 25113404
23  application.iOS                      0x0000000106646a60 0x104d98000 + 25881184
24  application.iOS                      0x000000010664c34c 0x104d98000 + 25903948
25  application.iOS                      0x0000000106694b44 0x104d98000 + 26200900
26  application.iOS                      0x00000001066948d0 0x104d98000 + 26200272
27  libsystem_pthread.dylib              0x00000001d2aabcb0 _pthread_start + 316
28  libsystem_pthread.dylib              0x00000001d2ab4778 thread_start + 4

Thread 8:
0   libsystem_kernel.dylib               0x00000001b4f52184 __workq_kernreturn + 8
1   libsystem_pthread.dylib              0x00000001d2ab476c start_wqthread + 4

Thread 9:
0   libsystem_kernel.dylib               0x00000001b4f52184 __workq_kernreturn + 8
1   libsystem_pthread.dylib              0x00000001d2ab476c start_wqthread + 4

Thread 10:
0   libsystem_kernel.dylib               0x00000001b4f52184 __workq_kernreturn + 8
1   libsystem_pthread.dylib              0x00000001d2ab476c start_wqthread + 4

Thread 11:
0   libsystem_kernel.dylib               0x00000001b4f52184 __workq_kernreturn + 8
1   libsystem_pthread.dylib              0x00000001d2ab476c start_wqthread + 4

Thread 12:
0   libsystem_kernel.dylib               0x00000001b4f532b4 kevent_qos + 8
1   libdispatch.dylib                    0x0000000186c15d4c _dispatch_kq_drain + 124
2   libdispatch.dylib                    0x0000000186c1578c _dispatch_event_loop_poke + 160
3   CFNetwork                            0x00000001876066dc 0x187600000 + 26332
4   CFNetwork                            0x000000018760507c 0x187600000 + 20604
5   CFNetwork                            0x000000018761e070 CFNetServiceBrowserSearchForServices + 76176
6   CFNetwork                            0x000000018761df30 CFNetServiceBrowserSearchForServices + 75856
7   CFNetwork                            0x0000000187601a88 0x187600000 + 6792
8   CFNetwork                            0x00000001878640bc __CFTubeSetTubeTypeNotifier + 59180
9   libdispatch.dylib                    0x0000000186bf424c _dispatch_call_block_and_release + 28
10  libdispatch.dylib                    0x0000000186bf5db0 _dispatch_client_callout + 16
11  libdispatch.dylib                    0x0000000186bfd10c _dispatch_lane_serial_drain + 576
12  libdispatch.dylib                    0x0000000186bfdc90 _dispatch_lane_invoke + 456
13  libdispatch.dylib                    0x0000000186c07d78 _dispatch_workloop_worker_thread + 704
14  libsystem_pthread.dylib              0x00000001d2aad814 _pthread_wqthread + 272
15  libsystem_pthread.dylib              0x00000001d2ab476c start_wqthread + 4

Thread 13:
0   libsystem_kernel.dylib               0x00000001b4f52184 __workq_kernreturn + 8
1   libsystem_pthread.dylib              0x00000001d2ab476c start_wqthread + 4

Thread 14:
0   libsystem_kernel.dylib               0x00000001b4f52184 __workq_kernreturn + 8
1   libsystem_pthread.dylib              0x00000001d2ab476c start_wqthread + 4

Thread 15:
0   libsystem_kernel.dylib               0x00000001b4f511ac __psynch_cvwait + 8
1   application.iOS                      0x00000001066f9b60 0x104d98000 + 26614624
2   application.iOS                      0x0000000106702840 0x104d98000 + 26650688
3   application.iOS                      0x0000000106688c2c 0x104d98000 + 26151980
4   application.iOS                      0x0000000106694a4c 0x104d98000 + 26200652
5   application.iOS                      0x00000001066948d0 0x104d98000 + 26200272
6   libsystem_pthread.dylib              0x00000001d2aabcb0 _pthread_start + 316
7   libsystem_pthread.dylib              0x00000001d2ab4778 thread_start + 4

Thread 16:
0   libsystem_kernel.dylib               0x00000001b4f52184 __workq_kernreturn + 8
1   libsystem_pthread.dylib              0x00000001d2ab476c start_wqthread + 4

Thread 17:
0   libsystem_kernel.dylib               0x00000001b4f52184 __workq_kernreturn + 8
1   libsystem_pthread.dylib              0x00000001d2ab476c start_wqthread + 4

Thread 18:
0   libsystem_kernel.dylib               0x00000001b4f52184 __workq_kernreturn + 8
1   libsystem_pthread.dylib              0x00000001d2ab476c start_wqthread + 4

Thread 19:
0   libsystem_trace.dylib                0x000000019dddfa04 _os_log_impl_flatten_and_send + 1764
1   libnetwork.dylib                     0x0000000187ab2c88 nw_endpoint_flow_setup_protocols + 3172
2   libnetwork.dylib                     0x0000000187ab07cc nw_endpoint_flow_setup_channel + 2820
3   libnetwork.dylib                     0x0000000187aacc00 -[NWConcrete_nw_endpoint_flow updatePathWithHandler:] + 3120
4   libnetwork.dylib                     0x0000000187bafc50 nw_endpoint_handler_path_change + 14588
5   libnetwork.dylib                     0x0000000187baada0 __nw_endpoint_handler_initialize_association_block_invoke + 152
6   libnetwork.dylib                     0x0000000187d3788c __nw_association_update_paths_block_invoke.81 + 88
7   libnetwork.dylib                     0x0000000187ee1b78 nw_hash_table_apply + 452
8   libnetwork.dylib                     0x0000000187d371b0 nw_association_update_paths + 484
9   libnetwork.dylib                     0x000000018803c508 nw_path_necp_update_evaluator + 2080
10  libnetwork.dylib                     0x000000018803b9b8 nw_path_necp_check_for_updates + 1020
11  libnetwork.dylib                     0x0000000188051f18 nw_path_evaluator_force_update + 124
12  libnetwork.dylib                     0x0000000187d3f0b4 nw_association_force_update + 104
13  libnetwork.dylib                     0x0000000187ab1b9c nw_endpoint_flow_setup_channel + 7892
14  libnetwork.dylib                     0x0000000187abfe44 -[NWConcrete_nw_endpoint_flow startWithHandler:] + 3732
15  libnetwork.dylib                     0x0000000187bafc50 nw_endpoint_handler_path_change + 14588
16  libnetwork.dylib                     0x0000000187bb6e14 nw_endpoint_handler_start + 1172
17  libnetwork.dylib                     0x0000000187ee89b0 nw_endpoint_resolver_start_next_child + 4476
18  libdispatch.dylib                    0x0000000186bf424c _dispatch_call_block_and_release + 28
19  libdispatch.dylib                    0x0000000186bf5db0 _dispatch_client_callout + 16
20  libdispatch.dylib                    0x0000000186bfef90 _dispatch_workloop_invoke + 1888
21  libdispatch.dylib                    0x0000000186c07d78 _dispatch_workloop_worker_thread + 704
22  libsystem_pthread.dylib              0x00000001d2aad814 _pthread_wqthread + 272
23  libsystem_pthread.dylib              0x00000001d2ab476c start_wqthread + 4

Thread 21:
0   libsystem_kernel.dylib               0x00000001b4f51814 __semwait_signal + 8
1   libsystem_c.dylib                    0x000000019042c5d4 usleep + 64
2   AVFCore                              0x000000018ebf419c -[AVFigRouteDiscovererOutputDeviceDiscoverySessionImpl _routePresentChanged] + 28
3   Foundation                           0x0000000188249490 -[__NSObserver _doit:] + 344
4   CoreFoundation                       0x0000000186f5e9a0 __CFNOTIFICATIONCENTER_IS_CALLING_OUT_TO_AN_OBSERVER__ + 24
5   CoreFoundation                       0x0000000186f5e948 ___CFXRegistrationPost_block_invoke + 48
6   CoreFoundation                       0x0000000186f5deb0 _CFXRegistrationPost + 436
7   CoreFoundation                       0x0000000186f5d870 _CFXNotificationPost + 712
8   CoreFoundation                       0x0000000186f5e4a8 CFNotificationCenterPostNotificationWithOptions + 132
9   CoreMedia                            0x000000018fe86ad8 CMNotificationCenterPostNotification + 124
10  CoreMedia                            0x000000018ff4650c __figXPC_HandleNotificationMessage_block_invoke + 188
11  libdispatch.dylib                    0x0000000186bf424c _dispatch_call_block_and_release + 28
12  libdispatch.dylib                    0x0000000186bf5db0 _dispatch_client_callout + 16
13  libdispatch.dylib                    0x0000000186bfd10c _dispatch_lane_serial_drain + 576
14  libdispatch.dylib                    0x0000000186bfdc5c _dispatch_lane_invoke + 404
15  libdispatch.dylib                    0x0000000186c07d78 _dispatch_workloop_worker_thread + 704
16  libsystem_pthread.dylib              0x00000001d2aad814 _pthread_wqthread + 272
17  libsystem_pthread.dylib              0x00000001d2ab476c start_wqthread + 4

Thread 22:
0   libsystem_kernel.dylib               0x00000001b4f52184 __workq_kernreturn + 8
1   libsystem_pthread.dylib              0x00000001d2ab476c start_wqthread + 4

Thread 23:
0   libsystem_kernel.dylib               0x00000001b4f52184 __workq_kernreturn + 8
1   libsystem_pthread.dylib              0x00000001d2ab476c start_wqthread + 4

Thread 24:
0   libsystem_kernel.dylib               0x00000001b4f52184 __workq_kernreturn + 8
1   libsystem_pthread.dylib              0x00000001d2ab476c start_wqthread + 4

Thread 25:
0   libsystem_kernel.dylib               0x00000001b4f52184 __workq_kernreturn + 8
1   libsystem_pthread.dylib              0x00000001d2ab476c start_wqthread + 4

Thread 26:
0   application.iOS                      0x00000001065637bc 0x104d98000 + 24950716
1   application.iOS                      0x000000010655e468 0x104d98000 + 24929384
2   application.iOS                      0x000000010655f5d0 0x104d98000 + 24933840
3   application.iOS                      0x000000010655ee0c 0x104d98000 + 24931852
4   application.iOS                      0x00000001065865fc 0x104d98000 + 25093628
5   application.iOS                      0x000000010658b050 0x104d98000 + 25112656
6   application.iOS                      0x000000010664837c 0x104d98000 + 25887612
7   application.iOS                      0x000000010664704c 0x104d98000 + 25882700
8   application.iOS                      0x0000000106586658 0x104d98000 + 25093720
9   application.iOS                      0x000000010658db30 0x104d98000 + 25123632
10  application.iOS                      0x000000010658d650 0x104d98000 + 25122384
11  application.iOS                      0x00000001051b602c 0x104d98000 + 4317228
12  application.iOS                      0x0000000104faf348 0x104d98000 + 2192200
13  application.iOS                      0x00000001060a3f70 0x104d98000 + 19971952
14  application.iOS                      0x0000000104f0c4e0 0x104d98000 + 1524960
15  application.iOS                      0x0000000104f0c1f4 0x104d98000 + 1524212
16  application.iOS                      0x0000000104f0c468 0x104d98000 + 1524840
17  application.iOS                      0x0000000104ef670c 0x104d98000 + 1435404
18  application.iOS                      0x0000000104ef650c 0x104d98000 + 1434892
19  application.iOS                      0x0000000104f10208 0x104d98000 + 1540616
20  application.iOS                      0x0000000104f0c380 0x104d98000 + 1524608
21  application.iOS                      0x0000000104f0c258 0x104d98000 + 1524312
22  application.iOS                      0x0000000104efb818 0x104d98000 + 1456152
23  application.iOS                      0x0000000105cc8ff0 0x104d98000 + 15929328
24  application.iOS                      0x0000000104efd8d8 0x104d98000 + 1464536
25  application.iOS                      0x0000000105178a00 0x104d98000 + 4065792
26  application.iOS                      0x000000010658b33c 0x104d98000 + 25113404
27  application.iOS                      0x000000010664837c 0x104d98000 + 25887612
28  application.iOS                      0x000000010668b9c8 0x104d98000 + 26163656
29  application.iOS                      0x00000001066892a4 0x104d98000 + 26153636
30  application.iOS                      0x0000000106694a4c 0x104d98000 + 26200652
31  application.iOS                      0x00000001066948d0 0x104d98000 + 26200272
32  libsystem_pthread.dylib              0x00000001d2aabcb0 _pthread_start + 316
33  libsystem_pthread.dylib              0x00000001d2ab4778 thread_start + 4

Thread 27:
0   libsystem_kernel.dylib               0x00000001b4f2d324 semaphore_timedwait_trap + 8
1   application.iOS                      0x0000000106694a4c 0x104d98000 + 26200652
2   application.iOS                      0x00000001066948d0 0x104d98000 + 26200272
3   libsystem_pthread.dylib              0x00000001d2aabcb0 _pthread_start + 316
4   libsystem_pthread.dylib              0x00000001d2ab4778 thread_start + 4

Thread 28:
0   libsystem_kernel.dylib               0x00000001b4f2d324 semaphore_timedwait_trap + 8
1   application.iOS                      0x0000000106694a4c 0x104d98000 + 26200652
2   application.iOS                      0x00000001066948d0 0x104d98000 + 26200272
3   libsystem_pthread.dylib              0x00000001d2aabcb0 _pthread_start + 316
4   libsystem_pthread.dylib              0x00000001d2ab4778 thread_start + 4

Thread 29:
0   libsystem_kernel.dylib               0x00000001b4f2d324 semaphore_timedwait_trap + 8
1   application.iOS                      0x0000000106694a4c 0x104d98000 + 26200652
2   application.iOS                      0x00000001066948d0 0x104d98000 + 26200272
3   libsystem_pthread.dylib              0x00000001d2aabcb0 _pthread_start + 316
4   libsystem_pthread.dylib              0x00000001d2ab4778 thread_start + 4

Thread 30:
0   libsystem_kernel.dylib               0x00000001b4f2d324 semaphore_timedwait_trap + 8
1   application.iOS                      0x0000000106694a4c 0x104d98000 + 26200652
2   application.iOS                      0x00000001066948d0 0x104d98000 + 26200272
3   libsystem_pthread.dylib              0x00000001d2aabcb0 _pthread_start + 316
4   libsystem_pthread.dylib              0x00000001d2ab4778 thread_start + 4

Thread 31:
0   libsystem_kernel.dylib               0x00000001b4f2d324 semaphore_timedwait_trap + 8
1   application.iOS                      0x0000000106694a4c 0x104d98000 + 26200652
2   application.iOS                      0x00000001066948d0 0x104d98000 + 26200272
3   libsystem_pthread.dylib              0x00000001d2aabcb0 _pthread_start + 316
4   libsystem_pthread.dylib              0x00000001d2ab4778 thread_start + 4

Thread 20 crashed with arm64 Thread State:
   x21: 0x000000016c4530e0     x2: 0x0000000000000000    x16: 0x0000000000000148     x3: 0x0000000000000000
   x22: 0x00000001082059e0     x4: 0x00000000000001b0    x17: 0x000000016c453000   cpsr: 0x0000000040000000
    x5: 0x00000001082ada10    x23: 0x0000000109826848     x6: 0x0000000000000021    x18: 0x0000000000000000
   x10: 0x00000000000bcfa0     lr: 0x00000001d2aacb50     x7: 0x000000028307dc70    x24: 0x000000010a41acc8
   x11: 0x0000000000003fff     x8: 0x00000000000005b9    x19: 0x0000000000000006    x25: 0x00000001084698f8
    x9: 0x0d05def64a16a7e2    x12: 0x0000000108200000     fp: 0x000000016c4511b0    x26: 0x000000010887b000
   x13: 0x00000000cb5b701f     pc: 0x00000001b4f51414    x27: 0x0000000000000002    x14: 0x00000000cb5b6ff8
   x20: 0x0000000000014627     x0: 0x0000000000000000     sp: 0x000000016c451190    x28: 0x00000001051789ff
   x15: 0x00000000000027aa     x1: 0x0000000000000000

Binary Images:
0x0000000104d98000 -        0x000000010683ffff +application.iOS arm64  <4ff40b39b7f93b72b2e36f63c52b1e81> /private/var/containers/Bundle/Application/B99CA9E0-70E0-4203-9EA3-7EB8B6B892B1/application.iOS.app/application.iOS
0x0000000186bf2000 -        0x0000000186c34fff  libdispatch.dylib arm64e  <b92757b104343e9ba3acb6da6b81945c> /usr/lib/system/libdispatch.dylib
0x0000000186c35000 -        0x0000000186c6ffff  libdyld.dylib arm64e  <21b19919133438bcb233896e929945e0> /usr/lib/system/libdyld.dylib
0x0000000186edc000 -        0x0000000187295fff  CoreFoundation arm64e  <727f2644eb4e3d57bc2ee6803ba92366> /System/Library/Frameworks/CoreFoundation.framework/CoreFoundation
0x0000000187600000 -        0x0000000187a97fff  CFNetwork arm64e  <0ebb3119a8ce35da96245b5528802fc8> /System/Library/Frameworks/CFNetwork.framework/CFNetwork
0x0000000187a98000 -        0x00000001881a5fff  libnetwork.dylib arm64e  <39506b46189e3e828f32ab8326fcc2c7> /usr/lib/libnetwork.dylib
0x000000018821d000 -        0x00000001884d8fff  Foundation arm64e  <7698bf3e0cf631c085e9562714f01276> /System/Library/Frameworks/Foundation.framework/Foundation
0x0000000188dee000 -        0x000000018a29bfff  UIKitCore arm64e  <8518eae3832b3ff09fa59dbe3041f26c> /System/Library/PrivateFrameworks/UIKitCore.framework/UIKitCore
0x000000018eab2000 -        0x000000018eacafff  AudioSession arm64e  <ec9f661dcafc3a16b372bb4a803636e8> /System/Library/PrivateFrameworks/AudioSession.framework/AudioSession
0x000000018eacb000 -        0x000000018ec6bfff  AVFCore arm64e  <0294540d081033fdbad051943cb01d86> /System/Library/PrivateFrameworks/AVFCore.framework/AVFCore
0x000000018fe69000 -        0x000000018ff69fff  CoreMedia arm64e  <063231877fe13d2d9c430fdad845e776> /System/Library/Frameworks/CoreMedia.framework/CoreMedia
0x00000001903b9000 -        0x0000000190437fff  libsystem_c.dylib arm64e  <961a8fb0de8a3567b4375d99cd549fae> /usr/lib/system/libsystem_c.dylib
0x000000019ddd1000 -        0x000000019dde7fff  libsystem_trace.dylib arm64e  <3520924c65a7351dadbea07268a9bca7> /usr/lib/system/libsystem_trace.dylib
0x000000019eb3d000 -        0x000000019eb45fff  GraphicsServices arm64e  <1959c615fa0830de8461884a5bbc8037> /System/Library/PrivateFrameworks/GraphicsServices.framework/GraphicsServices
0x00000001b4f29000 -        0x00000001b4f5afff  libsystem_kernel.dylib arm64e  <b373b97b671e301ca6b7c9f92e17dca0> /usr/lib/system/libsystem_kernel.dylib
0x00000001d2aaa000 -        0x00000001d2ab5fff  libsystem_pthread.dylib arm64e  <c939a2abd3b13b0a83c57ce2f4f339a9> /usr/lib/system/libsystem_pthread.dylib

mandel-macaque commented 3 years ago

@adams-family so, from the version you have in the App Store, you have 14.14.2.5 with hash 3836759d4, looking in the repo, the code of that version points to https://github.com/xamarin/xamarin-macios/blob/3836759d4/src/Foundation/NSUrlSessionHandler.cs#L543 which does not have the fix. Please update and get back to us.

adams-family commented 3 years ago

@mandel-macaque Got it! Updating and will push a new version to the AppStore. I should know in about it week if it helped.

spouliot commented 3 years ago

re-opening issue so this gets reviewed in 30 days (if not earlier) thanks!

adams-family commented 3 years ago

@spouliot I might have some news, not necessarily good one...

I deployed version 1.1.8 of my app with the latest stable Xamarin.iOS framework:

Xamarin.iOS
Version: 14.20.0.1 (Visual Studio Community)
Hash: fe0e2c518
Branch: d16-10
Build date: 2021-05-19 08:15:56-0400

But still got the same exception from production (AppStore) which afterwards ended up with a crash:

Not sure if I can get you a more detailed report than this one:

BTW: Is there a way I can retrieve the Xamarin.iOS version (hash) during runtime? I would pass it to AppCenter crashlytics alongside with the app version. The only hackaround I found is this but I don't find it particularly reliable: https://stackoverflow.com/questions/54288683/how-to-programmatically-retrieve-the-version-information-of-visual-studio-for-ma

spouliot commented 3 years ago

But still got the same exception from production (AppStore) which afterwards ended up with a crash:

@mandel-macaque ^

Is there a way I can retrieve the Xamarin.iOS version (hash) during runtime?

Not really - but you should not as knowing the version only gives you a very limited amount of information, which is better than nothing but we settle for less ;-)

I suggest you to generate msbuild binary logs (binlog files) when creating builds for submission. They are highly compressed, includes Xamarin.iOS version and a lot more very useful information. Keep the binlog along with your submitted .ipa and .dSYM.

That way you can read [1] it (or share it with us) in the future and figure out a lot more than just the SDK version.

[1] https://msbuildlog.com

rolfbjarne commented 3 years ago

Is there a way I can retrieve the Xamarin.iOS version (hash) during runtime?

Not really - but you should not as knowing the version only gives you a very limited amount of information, which is better than nothing but we settle for less ;-)

Actually yes, we embed the Xamarin.iOS version (including hash) into the Info.plist when building the app.

This is from an app I just on my hard drive:

$ more bin/iPhone/Debug/testapp.app/Info.plist | grep com.xamarin.ios -A 4
    <key>com.xamarin.ios</key>
    <dict>
        <key>Version</key>
        <string>14.9.0.143 (main: 14ad1a122)</string>
    </dict>

I suggest you to generate msbuild binary logs (binlog files) when creating builds for submission.

I heartily second this, the binlogs contain a wealth of information and are very useful for us in diagnosing all sorts of problems.

mandel-macaque commented 3 years ago

Lets try to see the version used and I'l dig deeper into the issue.

adams-family commented 3 years ago

@rolfbjarne Thanks, the version number embedded in Info.plist is great, this way it can be forwarded to crash reporting 👍

@mandel-macaque Confirm that the version is 14.20.0.1 (d16-10: fe0e2c518) and my entire Info.plist is below:

@spouliot "I suggest you to generate msbuild binary logs (binlog files)" - I agree, I did some research, can you please confirm to me that the following way is the right way to build binary logs on VS for Mac:

msbuild -bl

spouliot commented 3 years ago

to build binary logs on VS for Mac:

Not sure you can do them from VS for Mac. You normally want to be able to read build logs if you're in an IDE.

However you can build your .ipa from your solution on the command-line, e.g.

msbuild /t:Build /p:Platform=iPhone /p:Configuration=Release /bl:app.binlog app.sln

and you can use altool (from Apple) to automate the validation and submission of your .ipa to App Store Connect. Script it once and re-use multiple times :-)

mandel-macaque commented 3 years ago

@adams-family it is indeed an issue since the changes in that hash are present: https://github.com/xamarin/xamarin-macios/blob/fe0e2c5184ecfe9508607ebd90e39d9470d0515f/src/Foundation/NSUrlSessionHandler.cs#L544

adams-family commented 3 years ago

@mandel-macaque I just double checked that my app version 1.1.8 (188) was indeed built using Xamarin.iOS 14.20.0.1 (d16-10: fe0e2c518) - by going to Xcode -> Organizer -> find 1.1.8 (188) -> Show in Finder -> Show package contents -> Find Info.plist -> check com.xamarin.ios

The crash reason SIGABRT: Object reference not set to an instance of an object maybe signalizes another null reference problem?

mandel-macaque commented 3 years ago

@adams-family yes, because this has not been that common until now, there might be changes in iOS that are generating a race condition. Do you know how to reproduce it? Can you show me how you instantiate the HttpClient?

adams-family commented 3 years ago

@mandel-macaque Confirming that with the latest version of our app being automatically deployed by the AppStore the number of exceptions increases, so the problem still persists.

Sending samples of HttpClient usage below:

1) We use new HttpClient instances in a background thread like this - with a GET request:

using System;
using System.Net.Http;
using System.Threading.Tasks;

namespace HttpTest
{
    class HttpTestPlain
    {
        protected HttpClient client = new HttpClient();

        void Test()
        {
            var url = "https://httpbin.org/get";     // we don't actually use this url - this is just an example

            Task.Run(async () =>
            {
                try
                {
                    using (var client = new HttpClient())
                    {
                        using (var response = await client.GetAsync(url))
                        {
                            Console.WriteLine("Response code is: " + response.StatusCode);
                        }
                    }
                }
                catch (Exception e)
                {
                    Console.WriteLine(e.ToString());
                }
            });
        }
    }
}

2) We also use the Polly library for error ressiliance, also in a background thread asynchronously and then it looks like this - with a POST request:

using System;
using System.Collections.Generic;
using System.Net.Http;
using System.Threading.Tasks;
using Polly;

namespace HttpTest
{
    class HttpTestWithPolly
    {
        protected HttpClient client = new HttpClient();

        void Test()
        {
            var url = "https://httpbin.org/post";     // we don't actually use this url - this is just an example

            Task.Run(async () =>
            {
                try
                {
                    var postData = new List<KeyValuePair<string, string>>();
                    postData.Add(new KeyValuePair<string, string>("v", "1"));

                    HttpContent content = new FormUrlEncodedContent(postData);

                    var result = await Policy
                        .Handle<HttpRequestException>(ex => !ex.Message.ToLower().Contains("404"))
                        .WaitAndRetryAsync
                        (
                            retryCount: 3,
                            sleepDurationProvider: retryAttempt => TimeSpan.FromSeconds(Math.Pow(2, retryAttempt)),
                            onRetry: (ex, time) =>
                            {
                                Console.WriteLine("Something went wrong: {ex.Message}");
                            }
                        )
                        .ExecuteAsync(async () =>
                        {
                            var response = await client.PostAsync(url, content);

                            Console.WriteLine("Response code is: " + response.StatusCode);
                            return true;
                        });
                }
                catch (Exception e)
                {
                    Console.WriteLine(e.ToString());
                }
            });
        }
    }
}

Unfortunately from the crash logs it's unclear which one causes the crash, however I can say that we do have more places using the Polly wrapper than those that don't.

adams-family commented 3 years ago

@mandel-macaque Confirming that it's still happening on:

Version: 14.20.0.12 (Visual Studio Community)
Hash: bb11ec4c3
Branch: d16-10
Build date: 2021-06-09 10:47:24-0400

Stacktrace:

UNHANDLED (UnhandledException)
System.ObjectDisposedException: The CancellationTokenSource has been disposed.
 at System.Threading.CancellationTokenSource.Cancel (System.Boolean throwOnFirstException) <0x100acbba0 + 0x00038> in <da8885cdf78b449d96de00cdb9d47225#3eb8ccfc4f6758322994486ee103c98d>:0 
 at System.Threading.CancellationTokenSource.Cancel () <0x100acbb70 + 0x00017> in <da8885cdf78b449d96de00cdb9d47225#3eb8ccfc4f6758322994486ee103c98d>:0 
 at System.Net.Http.NSUrlSessionHandler+NSUrlSessionHandlerDelegate.DidCompleteWithError (Foundation.NSUrlSession session, Foundation.NSUrlSessionTask task, Foundation.NSError error) <0x10189f8a0 + 0x0007b> in <2e439eb70c714f5f9c50aa9db7939d35#3eb8ccfc4f6758322994486ee103c98d>:0  /   at System.Threading.CancellationTokenSource.Cancel (System.Boolean throwOnFirstException) <0x100acbba0 + 0x00038> in <da8885cdf78b449d96de00cdb9d47225#3eb8ccfc4f6758322994486ee103c98d>:0 
 at System.Threading.CancellationTokenSource.Cancel () <0x100acbb70 + 0x00017> in <da8885cdf78b449d96de00cdb9d47225#3eb8ccfc4f6758322994486ee103c98d>:0 
 at System.Net.Http.NSUrlSessionHandler+NSUrlSessionHandlerDelegate.DidCompleteWithError (Foundation.NSUrlSession session, Foundation.NSUrlSessionTask task, Foundation.NSError error) <0x10189f8a0 + 0x0007b> in <2e439eb70c714f5f9c50aa9db7939d35#3eb8ccfc4f6758322994486ee103c98d>:0  => System.ObjectDisposedException: The CancellationTokenSource has been disposed.
 at System.Threading.CancellationTokenSource.Cancel (System.Boolean throwOnFirstException) <0x100acbba0 + 0x00038> in <da8885cdf78b449d96de00cdb9d47225#3eb8ccfc4f6758322994486ee103c98d>:0 
 at System.Threading.CancellationTokenSource.Cancel () <0x100acbb70 + 0x00017> in <da8885cdf78b449d96de00cdb9d47225#3eb8ccfc4f6758322994486ee103c98d>:0 
 at System.Net.Http.NSUrlSessionHandler+NSUrlSessionHandlerDelegate.DidCompleteWithError (Foundation.NSUrlSession session, Foundation.NSUrlSessionTask task, Foundation.NSError error) <0x10189f8a0 + 0x0007b> in <2e439eb70c714f5f9c50aa9db7939d35#3eb8ccfc4f6758322994486ee103c98d>:0

adams-family commented 3 years ago

@mandel-macaque Just confirming that I received a crash report from Xamarin.iOS 14.20.0.24 (d16-10: c4b89cddb):

System.NullReferenceException: Object reference not set to an instance of an object
 at System.Net.Http.NSUrlSessionHandler+NSUrlSessionHandlerDelegate.DidCompleteWithError (Foundation.NSUrlSession session, Foundation.NSUrlSessionTask task, Foundation.NSError error) <0x10123b8a0 + 0x00074>
  in <16c89537d3e84c9193e31188658ef0ac#5b5c7dd2729acb21ba638e5f62afdc5f>:0
    /   at System.Net.Http.NSUrlSessionHandler+NSUrlSessionHandlerDelegate.DidCompleteWithError (Foundation.NSUrlSession session, Foundation.NSUrlSessionTask task, Foundation.NSError error) <0x10123b8a0 + 0x00074>
  in <16c89537d3e84c9193e31188658ef0ac#5b5c7dd2729acb21ba638e5f62afdc5f>:0 
=> System.NullReferenceException: Object reference not set to an instance of an object
 at System.Net.Http.NSUrlSessionHandler+NSUrlSessionHandlerDelegate.DidCompleteWithError (Foundation.NSUrlSession session, Foundation.NSUrlSessionTask task, Foundation.NSError error) <0x10123b8a0 + 0x00074>
  in <16c89537d3e84c9193e31188658ef0ac#5b5c7dd2729acb21ba638e5f62afdc5f>:0

mandel-macaque commented 3 years ago

Hmmm we might have an issue with Polly, I di don't thing about that, i'll take a look with that exact sample.

SuperMario77 commented 3 years ago

@mandel-macaque This issue is still a big problem in our app. In #9132 you said there was a fix made in branch d16-10. One week ago we finally updated Xamarin.iOS to 14.20.0.24 in our app hoping the issue was fixed but I can confirm the issue still persists or partly persists. The exception in "DidCompleteWithError" still occurred with the latest Xamarin.iOS version.

We have installed this version of Xamarin.iOS in Visual Studio for Mac:

Xamarin.iOS
Version: 14.20.0.24 (Visual Studio Community)
Hash: c4b89cddb
Branch: d16-10
Build date: 2021-06-15 22:03:01-0400

This is a callstack from AppCenter of one of our users:

Date/Time:       2021-08-16T12:55:31.999Z
OS Version:      iPhone OS 14.7.1 (18G82)

Exception Type:  SIGABRT
Exception Codes: #0 at 0x1d3d2f334
Crashed Thread:  17

Application Specific Information:
*** Terminating app due to uncaught exception 'SIGABRT', reason: 'Object reference not set to an instance of an object'

Xamarin Exception Stack:
System.NullReferenceException: Object reference not set to an instance of an object
  at System.Net.Http.NSUrlSessionHandler+NSUrlSessionHandlerDelegate.DidCompleteWithError (Foundation.NSUrlSession session, Foundation.NSUrlSessionTask task, Foundation.NSError error) [0x00023] in /Users/builder/azdo/_work/1/s/xamarin-macios/src/Foundation/NSUrlSessionHandler.cs:674

https://github.com/xamarin/xamarin-macios/blob/d16-10/src/Foundation/NSUrlSessionHandler.cs The line 674 looks like this:

672:        if (error != null || serverError != null) {
673:            // got an error, cancel the stream operatios before we do anything
674:            inflight.CancellationTokenSource.Cancel ();       <------- crash line
675:            inflight.Errored = true;

So, I assume like in #9132 the CancellationTokenSource is null and therefore the crash. Why not check for null here?

BTW, why I can see the path of the developers workspace in official version 14.20.0.24? (/Users/builder/azdo/_work/....)

In the AppCenter callstacks of the older version of Xamarin.iOS 13.20.2.2 I saw the version string in the path: NSUrlSessionHandler+NSUrlSessionHandlerDelegate.DidCompleteWithError (Foundation.NSUrlSession session, Foundation.NSUrlSessionTask task, Foundation.NSError error) /Library/Frameworks/Xamarin.iOS.framework/Versions/13.20.2.2/src/Xamarin.iOS/Foundation/NSUrlSessionHandler.cs:663

[UPDATE] We don't use Polly in our app.

shushil commented 3 years ago

Hello guys I am having the same issue my app. I am seeing same crash logs on the Appcenter as SuperMario77 and adams-family is describing.

smileyiori commented 2 years ago

Hello guys, I am getting the same issues in different two places of my application.

Xamarin Exception Stack: System.NullReferenceException: Object reference not set to an instance of an object. at System.Net.Http.NSUrlSessionHandler+NSUrlSessionHandlerDelegate.DidCompleteWithError (Foundation.NSUrlSession session, Foundation.NSUrlSessionTask task, Foundation.NSError error) <0x1012aa408 + 0x00148> in <5b08e01cb6df409eb2fea153e6177e5d>:0

It happened in my main application and in my notification extension application. My notification extension sends a request to an api using this code for HttpClient:

NSUrlSessionConfiguration config = NSUrlSessionConfiguration.DefaultSessionConfiguration;

var crendentials = new NSUrlCredential(username, password, NSUrlCredentialPersistence.ForSession);

var uri = new Uri(ApiUrl);
var urlProtectionSpace = new NSUrlProtectionSpace(uri.Host, uri.Port, uri.Scheme, "my realm", NSUrlProtectionSpace.AuthenticationMethodHTTPBasic);
config.URLCredentialStorage.SetDefaultCredential(crendentials, urlProtectionSpace);

var httpClientHandler = new NSUrlSessionHandler(config);

var httpClient = new HttpClient(httpClientHandler)
{
  BaseUrl = ApiUrl
};

Also this occurs occasionally, it happened 3 times in 7 days. I have around 1,000 users.

I will follow @SuperMario77 and say that CancellationTokenSource could be null and it may be throwing an exception.

mandel-macaque commented 2 years ago

I am back looking at this bug (sorry, other work had to be completed first). After a careful analysis I want to throw some light into the issue so that people understand the implications of the exception and what is going on.

First, let me answer a question raised in the thread:

@SuperMario77 Why not check for null here?

The reason for me not adding a null check there is because in the manage land, there is nothing that would set the CancellationTokenSource to null and therefore if we do check for null there are simply hiding the real bug and we would not provide you with a robust fix, I would be only hiding the bug under the rug.

Root cause

Let me try to explain what we are seeing and how we will have to fix the bug. The main issue we are seeing is similar to and old bug in Xamarin and its runtime and is a consequence of the fact that Xamarin is bridging two different worlds, the managed world in C# and the unmanaged world in Objc. The original bug in Xamarin was reported here: https://bugzilla.xamarin.com/37/37670/bug.html in 2016. Rolf explains the theory of what was happening with is similar to what we are seeing: https://bugzilla.xamarin.com/37/37670/bug.html#c9

How is this happening currently, the main issue is due to the object tree that is created by the InflightData class, which is later "returned" by the Xamarin Runtime with all its managed fields set to null. If we look closely to the class we have the following:

        class InflightData : IDisposable
        {
            public readonly object Lock = new object ();
            public string RequestUrl { get; set; }

            public TaskCompletionSource<HttpResponseMessage> CompletionSource { get; } = new TaskCompletionSource<HttpResponseMessage> ();
            public CancellationToken CancellationToken { get; set; }
            public CancellationTokenSource CancellationTokenSource { get; } = new CancellationTokenSource ();
            public NSUrlSessionDataTaskStream Stream { get; } = new NSUrlSessionDataTaskStream ();
            public HttpRequestMessage Request { get; set; }
            public HttpResponseMessage? Response { get; set; }

            public Exception? Exception { get; set; }
            public bool ResponseSent { get; set; }
            public bool Errored { get; set; }
            public bool Disposed { get; set; }
            public bool Completed { get; set; }
            public bool Done { get { return Errored || Disposed || Completed || CancellationToken.IsCancellationRequested; } }

            // code continues but is not of interested for the bug

The above class contains a reference to a NSUrlSessionDataTaskStream which later contains a reference to a Stream implementation that has a pointer to a NSData. This pointer is the one I believe is making or runtime to get confused and behave in a similar manner to what was described in 2016. The NSData cannot be Disposed due to what is explained in a code comment:

// We cannot use current?.Dispose. The reason is the following one:
// In the DidReceiveResponse, if iOS realizes that a buffer can be reused,
// because the data is the same, it will do so. Such a situation does happen
// between requests, that is, request A and request B will get the same NSData
// (buffer) in the delegate. In this case, we cannot dispose the NSData because
// it might be that a different request received it and it is present in
// its NSUrlSessionDataTaskStream stream. We can only trust the gc to do the job
// which is better than copying the data over. 
current = null;
currentStream = null;

At this point and due to the multithread nature of the GC, Delegates and others we are in a situation in which we are getting a recreated InflighData object that has been init by the runtime but has fields that are null when the should not! That results in the CancellationTokenSource being null when nothing in the managed code is setting it to null (if you look closely into the InflightData implementation, the Dispose method DOES NOT set the token to null).

The way to fix this is to separate the cancelation token and the inflight data, so that if we find a null inflight data or any of its contents to be null, we can cancel the request. Doing this should also result is a smaller chance to hit that race condition in which we get a managed object that is a zombie.

I have already started working in the fix which will need to be tested before we release it. I will be targeting the latest stable release of d17-3.

chamons commented 2 years ago

Manuel has a potential fix in https://github.com/xamarin/xamarin-macios/pull/15678 however it will likely stay as a draft and not land/be released if don't have a sample project reproducing the bug in question.

These networking fixes are absurdly difficult to pin down, and without a way to validate our work it is hard to know if this fixes/improves behavior.

If you have interest in this work, please consider testing the package here and reporting back or better yet get us a sample project that shows the behavior. If we have multiple confirmations of improvement we might be able to land this without a sample for testing.

narciszait commented 1 year ago

hi. would it be possible to make a new package that has iOS 16 in it and the proposed fix? we are seeing a lot of crashes with this error and would like to try it out

rolfbjarne commented 1 year ago

@narciszait are you building with .NET or Xamarin.iOS?

narciszait commented 1 year ago

@rolfbjarne Xamarin.iOS

hjharvey-MSFT commented 1 year ago

@narciszait please let us know if this works. With confirmation we can roll this into general availability. This would be really helpful for us. Thanks!

narciszait commented 1 year ago

sure, i will. the problem is that it is very hard to replicate and i have seen a couple of issues opened about NSURLSessionHandlerDelegate with either NullRefException or CancellationToken - they seem very related. another slight problem is that with the holidays around, it will be a little hard to monitor and keep an eye out until after New Years

rolfbjarne commented 1 year ago

@narciszait can you try this package:

xamarin.ios-16.2.0.9.pkg

narciszait commented 1 year ago

@rolfbjarne sorry for the late reply - winter holidays. i will give it a try, monitor for some time and let you know

goelze commented 1 year ago

Is it possible to test the proposed fix on .NET 7 for iOS? For some reason, we are only seeing this error after migrating from Xamarin.iOS to .NET 7 for iOS.

rolfbjarne commented 1 year ago

@goelze unfortunately testing custom builds on .NET is a rather complex process compared to Xamarin.iOS, but I'll have a look and see if I can figure out what needs to be done.

goelze commented 1 year ago

@rolfbjarne I actually was able to download and build from source by applying the commits to the latest .NET 7 iOS release, and was able to install it locally using the Makefile. But I ran into an issue with the maui-ios workload conflicting with my locally built ios workload, so I was unable to build/run our project.

I guess I would also need to build the maui-ios workload from source as well?

rolfbjarne commented 1 year ago

@rolfbjarne I actually was able to download and build from source by applying the commits to the latest .NET 7 iOS release, and was able to install it locally using the Makefile. But I ran into an issue with the maui-ios workload conflicting with my locally built ios workload, so I was unable to build/run our project.

I guess I would also need to build the maui-ios workload from source as well?

Oh that's great!

Exactly which issue did you run into?

goelze commented 1 year ago

A minor correction, I actually built off of the release/7.0.1xx-xcode14.2 branch.

So after running make world followed by make dotnet-install-system, I try to install the maui-ios workload with sudo dotnet workload install maui-ios and get this output:

Console output

``` sudo dotnet workload install maui-ios Welcome to .NET 7.0! --------------------- SDK Version: 7.0.101 Telemetry --------- The .NET tools collect usage data in order to help us improve your experience. It is collected by Microsoft and shared with the community. You can opt-out of telemetry by setting the DOTNET_CLI_TELEMETRY_OPTOUT environment variable to '1' or 'true' using your favorite shell. Read more about .NET CLI Tools telemetry: https://aka.ms/dotnet-cli-telemetry ---------------- Installed an ASP.NET Core HTTPS development certificate. To trust the certificate run 'dotnet dev-certs https --trust' (Windows and macOS only). Learn about HTTPS: https://aka.ms/dotnet-https ---------------- Write your first app: https://aka.ms/dotnet-hello-world Find out what's new: https://aka.ms/dotnet-whats-new Explore documentation: https://aka.ms/dotnet-docs Report issues and find source on GitHub: https://github.com/dotnet/core Use 'dotnet --help' to see available commands or visit: https://aka.ms/dotnet-cli -------------------------------------------------------------------------------------- Skipping NuGet package signature verification. Skipping NuGet package signature verification. Skipping NuGet package signature verification. Skipping NuGet package signature verification. Skipping NuGet package signature verification. Installing pack Microsoft.Maui.Core.Ref.ios version 7.0.52... Writing workload pack installation record for Microsoft.Maui.Core.Ref.ios version 7.0.52... Installing pack Microsoft.Maui.Core.Runtime.ios version 7.0.52... Writing workload pack installation record for Microsoft.Maui.Core.Runtime.ios version 7.0.52... Installing pack Microsoft.Maui.Controls.Ref.ios version 7.0.52... Writing workload pack installation record for Microsoft.Maui.Controls.Ref.ios version 7.0.52... Installing pack Microsoft.Maui.Controls.Runtime.ios version 7.0.52... Writing workload pack installation record for Microsoft.Maui.Controls.Runtime.ios version 7.0.52... Installing pack Microsoft.Maui.Essentials.Ref.ios version 7.0.52... Writing workload pack installation record for Microsoft.Maui.Essentials.Ref.ios version 7.0.52... Installing pack Microsoft.Maui.Essentials.Runtime.ios version 7.0.52... Writing workload pack installation record for Microsoft.Maui.Essentials.Runtime.ios version 7.0.52... Installing pack Microsoft.AspNetCore.Components.WebView.Maui version 7.0.52... Writing workload pack installation record for Microsoft.AspNetCore.Components.WebView.Maui version 7.0.52... Installing pack Microsoft.Maui.Sdk version 7.0.52... Writing workload pack installation record for Microsoft.Maui.Sdk.net7 version 7.0.52... Installing pack Microsoft.Maui.Sdk version 6.0.548... Writing workload pack installation record for Microsoft.Maui.Sdk.net6 version 6.0.548... Installing pack Microsoft.Maui.Graphics version 7.0.52... Writing workload pack installation record for Microsoft.Maui.Graphics version 7.0.52... Installing pack Microsoft.Maui.Resizetizer.Sdk version 7.0.52... Writing workload pack installation record for Microsoft.Maui.Resizetizer.Sdk version 7.0.52... Installing pack Microsoft.Maui.Templates.net7 version 7.0.52... Writing workload pack installation record for Microsoft.Maui.Templates.net7 version 7.0.52... Installing pack Microsoft.Maui.Templates.net6 version 6.0.548... Writing workload pack installation record for Microsoft.Maui.Templates.net6 version 6.0.548... Installing pack Microsoft.Maui.Core.Ref.any version 7.0.52... Writing workload pack installation record for Microsoft.Maui.Core.Ref.any version 7.0.52... Installing pack Microsoft.Maui.Core.Runtime.any version 7.0.52... Writing workload pack installation record for Microsoft.Maui.Core.Runtime.any version 7.0.52... Installing pack Microsoft.Maui.Controls.Ref.any version 7.0.52... Writing workload pack installation record for Microsoft.Maui.Controls.Ref.any version 7.0.52... Installing pack Microsoft.Maui.Controls.Runtime.any version 7.0.52... Writing workload pack installation record for Microsoft.Maui.Controls.Runtime.any version 7.0.52... Installing pack Microsoft.Maui.Essentials.Ref.any version 7.0.52... Writing workload pack installation record for Microsoft.Maui.Essentials.Ref.any version 7.0.52... Installing pack Microsoft.Maui.Essentials.Runtime.any version 7.0.52... Writing workload pack installation record for Microsoft.Maui.Essentials.Runtime.any version 7.0.52... Installing pack Microsoft.iOS.Sdk version 16.2.1010-ci.test... Pack Microsoft.iOS.Sdk version 16.2.1010-ci.test is already installed. Writing workload pack installation record for Microsoft.iOS.Sdk.net7 version 16.2.1010-ci.test... Installing pack Microsoft.iOS.Sdk version 16.2.3... Workload installation failed. Rolling back installed packs... Rolling back pack Microsoft.iOS.Sdk installation... Rolling back pack Microsoft.Maui.Essentials.Runtime.any installation... Uninstalling workload pack Microsoft.Maui.Essentials.Runtime.any version 7.0.52… Rolling back pack Microsoft.Maui.Essentials.Ref.any installation... Uninstalling workload pack Microsoft.Maui.Essentials.Ref.any version 7.0.52… Rolling back pack Microsoft.Maui.Controls.Runtime.any installation... Uninstalling workload pack Microsoft.Maui.Controls.Runtime.any version 7.0.52… Rolling back pack Microsoft.Maui.Controls.Ref.any installation... Uninstalling workload pack Microsoft.Maui.Controls.Ref.any version 7.0.52… Rolling back pack Microsoft.Maui.Core.Runtime.any installation... Uninstalling workload pack Microsoft.Maui.Core.Runtime.any version 7.0.52… Rolling back pack Microsoft.Maui.Core.Ref.any installation... Uninstalling workload pack Microsoft.Maui.Core.Ref.any version 7.0.52… Rolling back pack Microsoft.Maui.Templates.net6 installation... Uninstalling workload pack Microsoft.Maui.Templates.net6 version 6.0.548… Rolling back pack Microsoft.Maui.Templates.net7 installation... Uninstalling workload pack Microsoft.Maui.Templates.net7 version 7.0.52… Rolling back pack Microsoft.Maui.Resizetizer.Sdk installation... Uninstalling workload pack Microsoft.Maui.Resizetizer.Sdk version 7.0.52… Rolling back pack Microsoft.Maui.Graphics installation... Uninstalling workload pack Microsoft.Maui.Graphics version 7.0.52… Rolling back pack Microsoft.Maui.Sdk installation... Uninstalling workload pack Microsoft.Maui.Sdk.net6 version 6.0.548… Rolling back pack Microsoft.Maui.Sdk installation... Uninstalling workload pack Microsoft.Maui.Sdk.net7 version 7.0.52… Rolling back pack Microsoft.AspNetCore.Components.WebView.Maui installation... Uninstalling workload pack Microsoft.AspNetCore.Components.WebView.Maui version 7.0.52… Rolling back pack Microsoft.Maui.Essentials.Runtime.ios installation... Uninstalling workload pack Microsoft.Maui.Essentials.Runtime.ios version 7.0.52… Rolling back pack Microsoft.Maui.Essentials.Ref.ios installation... Uninstalling workload pack Microsoft.Maui.Essentials.Ref.ios version 7.0.52… Rolling back pack Microsoft.Maui.Controls.Runtime.ios installation... Uninstalling workload pack Microsoft.Maui.Controls.Runtime.ios version 7.0.52… Rolling back pack Microsoft.Maui.Controls.Ref.ios installation... Uninstalling workload pack Microsoft.Maui.Controls.Ref.ios version 7.0.52… Rolling back pack Microsoft.Maui.Core.Runtime.ios installation... Uninstalling workload pack Microsoft.Maui.Core.Runtime.ios version 7.0.52… Rolling back pack Microsoft.Maui.Core.Ref.ios installation... Uninstalling workload pack Microsoft.Maui.Core.Ref.ios version 7.0.52… Workload installation failed: microsoft.ios.sdk::16.2.3 is not found in NuGet feeds https://api.nuget.org/v3/index.json". ```

I need the maui-ios workload because I have the <UseMauiEssentials>true</UseMauiEssentials> line in our csproj file, at least that's my assumption.

It looks like some version resolution issues, but I'm not entirely sure what's happening.

goelze commented 1 year ago

Ok, building off that branch actually seemed to be an issue. I re-built using xcode 14.1 and the latest .net 7 release commit, and installed it with make dotnet-install-system. When I run sudo dotnet workload install maui-ios, I get this output:

Console output

``` sudo dotnet workload install maui-ios Welcome to .NET 7.0! --------------------- SDK Version: 7.0.101 Telemetry --------- The .NET tools collect usage data in order to help us improve your experience. It is collected by Microsoft and shared with the community. You can opt-out of telemetry by setting the DOTNET_CLI_TELEMETRY_OPTOUT environment variable to '1' or 'true' using your favorite shell. Read more about .NET CLI Tools telemetry: https://aka.ms/dotnet-cli-telemetry ---------------- Installed an ASP.NET Core HTTPS development certificate. To trust the certificate run 'dotnet dev-certs https --trust' (Windows and macOS only). Learn about HTTPS: https://aka.ms/dotnet-https ---------------- Write your first app: https://aka.ms/dotnet-hello-world Find out what's new: https://aka.ms/dotnet-whats-new Explore documentation: https://aka.ms/dotnet-docs Report issues and find source on GitHub: https://github.com/dotnet/core Use 'dotnet --help' to see available commands or visit: https://aka.ms/dotnet-cli -------------------------------------------------------------------------------------- Skipping NuGet package signature verification. Installing pack Microsoft.Maui.Core.Ref.ios version 7.0.55... Workload installation failed. Rolling back installed packs... Rolling back pack Microsoft.Maui.Core.Ref.ios installation... Workload installation failed: microsoft.maui.core.ref.ios::7.0.55 is not found in NuGet feeds https://api.nuget.org/v3/index.json". ```

I noticed that when I run the same command, but in the xamarin-macios source folder, the installation actually works, with the following output:

Console output

``` sudo dotnet workload install maui-ios Welcome to .NET 7.0! --------------------- SDK Version: 7.0.101 Telemetry --------- The .NET tools collect usage data in order to help us improve your experience. It is collected by Microsoft and shared with the community. You can opt-out of telemetry by setting the DOTNET_CLI_TELEMETRY_OPTOUT environment variable to '1' or 'true' using your favorite shell. Read more about .NET CLI Tools telemetry: https://aka.ms/dotnet-cli-telemetry ---------------- Installed an ASP.NET Core HTTPS development certificate. To trust the certificate run 'dotnet dev-certs https --trust' (Windows and macOS only). Learn about HTTPS: https://aka.ms/dotnet-https ---------------- Write your first app: https://aka.ms/dotnet-hello-world Find out what's new: https://aka.ms/dotnet-whats-new Explore documentation: https://aka.ms/dotnet-docs Report issues and find source on GitHub: https://github.com/dotnet/core Use 'dotnet --help' to see available commands or visit: https://aka.ms/dotnet-cli -------------------------------------------------------------------------------------- Skipping NuGet package signature verification. Installing workload manifest microsoft.net.sdk.ios version 16.2.1004… Installing workload manifest microsoft.net.sdk.maccatalyst version 16.2.1004… Installing workload manifest microsoft.net.sdk.tvos version 16.1.1501… Installing pack Microsoft.Maui.Core.Ref.ios version 7.0.55... Writing workload pack installation record for Microsoft.Maui.Core.Ref.ios version 7.0.55... Installing pack Microsoft.Maui.Core.Runtime.ios version 7.0.55... Writing workload pack installation record for Microsoft.Maui.Core.Runtime.ios version 7.0.55... Installing pack Microsoft.Maui.Controls.Ref.ios version 7.0.55... Writing workload pack installation record for Microsoft.Maui.Controls.Ref.ios version 7.0.55... Installing pack Microsoft.Maui.Controls.Runtime.ios version 7.0.55... Writing workload pack installation record for Microsoft.Maui.Controls.Runtime.ios version 7.0.55... Installing pack Microsoft.Maui.Essentials.Ref.ios version 7.0.55... Writing workload pack installation record for Microsoft.Maui.Essentials.Ref.ios version 7.0.55... Installing pack Microsoft.Maui.Essentials.Runtime.ios version 7.0.55... Writing workload pack installation record for Microsoft.Maui.Essentials.Runtime.ios version 7.0.55... Installing pack Microsoft.AspNetCore.Components.WebView.Maui version 7.0.55... Writing workload pack installation record for Microsoft.AspNetCore.Components.WebView.Maui version 7.0.55... Installing pack Microsoft.Maui.Sdk version 7.0.55... Writing workload pack installation record for Microsoft.Maui.Sdk.net7 version 7.0.55... Installing pack Microsoft.Maui.Sdk version 6.0.550... Writing workload pack installation record for Microsoft.Maui.Sdk.net6 version 6.0.550... Installing pack Microsoft.Maui.Graphics version 7.0.55... Writing workload pack installation record for Microsoft.Maui.Graphics version 7.0.55... Installing pack Microsoft.Maui.Resizetizer.Sdk version 7.0.55... Writing workload pack installation record for Microsoft.Maui.Resizetizer.Sdk version 7.0.55... Installing pack Microsoft.Maui.Templates.net7 version 7.0.55... Writing workload pack installation record for Microsoft.Maui.Templates.net7 version 7.0.55... Installing pack Microsoft.Maui.Templates.net6 version 6.0.550... Writing workload pack installation record for Microsoft.Maui.Templates.net6 version 6.0.550... Installing pack Microsoft.Maui.Core.Ref.any version 7.0.55... Writing workload pack installation record for Microsoft.Maui.Core.Ref.any version 7.0.55... Installing pack Microsoft.Maui.Core.Runtime.any version 7.0.55... Writing workload pack installation record for Microsoft.Maui.Core.Runtime.any version 7.0.55... Installing pack Microsoft.Maui.Controls.Ref.any version 7.0.55... Writing workload pack installation record for Microsoft.Maui.Controls.Ref.any version 7.0.55... Installing pack Microsoft.Maui.Controls.Runtime.any version 7.0.55... Writing workload pack installation record for Microsoft.Maui.Controls.Runtime.any version 7.0.55... Installing pack Microsoft.Maui.Essentials.Ref.any version 7.0.55... Writing workload pack installation record for Microsoft.Maui.Essentials.Ref.any version 7.0.55... Installing pack Microsoft.Maui.Essentials.Runtime.any version 7.0.55... Writing workload pack installation record for Microsoft.Maui.Essentials.Runtime.any version 7.0.55... Installing pack Microsoft.iOS.Sdk version 16.2.1004... Writing workload pack installation record for Microsoft.iOS.Sdk.net7 version 16.2.1004... Installing pack Microsoft.iOS.Sdk version 16.2.3... Writing workload pack installation record for Microsoft.iOS.Sdk.net6 version 16.2.3... Installing pack Microsoft.iOS.Ref version 16.2.1004... Writing workload pack installation record for Microsoft.iOS.Ref version 16.2.1004... Installing pack Microsoft.iOS.Runtime.ios-arm version 16.2.1004... Writing workload pack installation record for Microsoft.iOS.Runtime.ios-arm version 16.2.1004... Installing pack Microsoft.iOS.Runtime.ios-arm64 version 16.2.1004... Writing workload pack installation record for Microsoft.iOS.Runtime.ios-arm64 version 16.2.1004... Installing pack Microsoft.iOS.Runtime.iossimulator-x86 version 16.2.1004... Writing workload pack installation record for Microsoft.iOS.Runtime.iossimulator-x86 version 16.2.1004... Installing pack Microsoft.iOS.Runtime.iossimulator-x64 version 16.2.1004... Writing workload pack installation record for Microsoft.iOS.Runtime.iossimulator-x64 version 16.2.1004... Installing pack Microsoft.iOS.Runtime.iossimulator-arm64 version 16.2.1004... Writing workload pack installation record for Microsoft.iOS.Runtime.iossimulator-arm64 version 16.2.1004... Installing pack Microsoft.iOS.Templates version 16.2.1004... Writing workload pack installation record for Microsoft.iOS.Templates.net7 version 16.2.1004... Installing pack Microsoft.NETCore.App.Runtime.AOT.osx-x64.Cross.ios-arm version 7.0.1... Writing workload pack installation record for Microsoft.NETCore.App.Runtime.AOT.Cross.net7.ios-arm version 7.0.1... Installing pack Microsoft.NETCore.App.Runtime.AOT.osx-x64.Cross.ios-arm64 version 7.0.1... Writing workload pack installation record for Microsoft.NETCore.App.Runtime.AOT.Cross.net7.ios-arm64 version 7.0.1... Installing pack Microsoft.NETCore.App.Runtime.AOT.osx-x64.Cross.iossimulator-arm64 version 7.0.1... Writing workload pack installation record for Microsoft.NETCore.App.Runtime.AOT.Cross.net7.iossimulator-arm64 version 7.0.1... Installing pack Microsoft.NETCore.App.Runtime.AOT.osx-x64.Cross.iossimulator-x64 version 7.0.1... Writing workload pack installation record for Microsoft.NETCore.App.Runtime.AOT.Cross.net7.iossimulator-x64 version 7.0.1... Installing pack Microsoft.NETCore.App.Runtime.AOT.osx-x64.Cross.iossimulator-x86 version 7.0.1... Writing workload pack installation record for Microsoft.NETCore.App.Runtime.AOT.Cross.net7.iossimulator-x86 version 7.0.1... Installing pack Microsoft.NETCore.App.Runtime.Mono.ios-arm version 7.0.1... Writing workload pack installation record for Microsoft.NETCore.App.Runtime.Mono.net7.ios-arm version 7.0.1... Installing pack Microsoft.NETCore.App.Runtime.Mono.ios-arm64 version 7.0.1... Writing workload pack installation record for Microsoft.NETCore.App.Runtime.Mono.net7.ios-arm64 version 7.0.1... Installing pack Microsoft.NETCore.App.Runtime.Mono.iossimulator-arm64 version 7.0.1... Writing workload pack installation record for Microsoft.NETCore.App.Runtime.Mono.net7.iossimulator-arm64 version 7.0.1... Installing pack Microsoft.NETCore.App.Runtime.Mono.iossimulator-x64 version 7.0.1... Writing workload pack installation record for Microsoft.NETCore.App.Runtime.Mono.net7.iossimulator-x64 version 7.0.1... Installing pack Microsoft.NETCore.App.Runtime.Mono.iossimulator-x86 version 7.0.1... Writing workload pack installation record for Microsoft.NETCore.App.Runtime.Mono.net7.iossimulator-x86 version 7.0.1... Installing pack Microsoft.NET.Runtime.MonoAOTCompiler.Task version 7.0.1... Writing workload pack installation record for Microsoft.NET.Runtime.MonoAOTCompiler.Task.net7 version 7.0.1... Installing pack Microsoft.NET.Runtime.MonoTargets.Sdk version 7.0.1... Writing workload pack installation record for Microsoft.NET.Runtime.MonoTargets.Sdk.net7 version 7.0.1... Installing pack Microsoft.NETCore.App.Runtime.AOT.osx-x64.Cross.ios-arm version 6.0.12... Writing workload pack installation record for Microsoft.NETCore.App.Runtime.AOT.Cross.net6.ios-arm version 6.0.12... Installing pack Microsoft.NETCore.App.Runtime.AOT.osx-x64.Cross.ios-arm64 version 6.0.12... Writing workload pack installation record for Microsoft.NETCore.App.Runtime.AOT.Cross.net6.ios-arm64 version 6.0.12... Installing pack Microsoft.NETCore.App.Runtime.AOT.osx-x64.Cross.iossimulator-arm64 version 6.0.12... Writing workload pack installation record for Microsoft.NETCore.App.Runtime.AOT.Cross.net6.iossimulator-arm64 version 6.0.12... Installing pack Microsoft.NETCore.App.Runtime.AOT.osx-x64.Cross.iossimulator-x64 version 6.0.12... Writing workload pack installation record for Microsoft.NETCore.App.Runtime.AOT.Cross.net6.iossimulator-x64 version 6.0.12... Installing pack Microsoft.NETCore.App.Runtime.AOT.osx-x64.Cross.iossimulator-x86 version 6.0.12... Writing workload pack installation record for Microsoft.NETCore.App.Runtime.AOT.Cross.net6.iossimulator-x86 version 6.0.12... Installing pack Microsoft.NETCore.App.Runtime.Mono.ios-arm version 6.0.12... Writing workload pack installation record for Microsoft.NETCore.App.Runtime.Mono.net6.ios-arm version 6.0.12... Installing pack Microsoft.NETCore.App.Runtime.Mono.ios-arm64 version 6.0.12... Writing workload pack installation record for Microsoft.NETCore.App.Runtime.Mono.net6.ios-arm64 version 6.0.12... Installing pack Microsoft.NETCore.App.Runtime.Mono.iossimulator-arm64 version 6.0.12... Writing workload pack installation record for Microsoft.NETCore.App.Runtime.Mono.net6.iossimulator-arm64 version 6.0.12... Installing pack Microsoft.NETCore.App.Runtime.Mono.iossimulator-x64 version 6.0.12... Writing workload pack installation record for Microsoft.NETCore.App.Runtime.Mono.net6.iossimulator-x64 version 6.0.12... Installing pack Microsoft.NETCore.App.Runtime.Mono.iossimulator-x86 version 6.0.12... Writing workload pack installation record for Microsoft.NETCore.App.Runtime.Mono.net6.iossimulator-x86 version 6.0.12... Installing pack Microsoft.NET.Runtime.MonoAOTCompiler.Task version 6.0.12... Writing workload pack installation record for Microsoft.NET.Runtime.MonoAOTCompiler.Task.net6 version 6.0.12... Installing pack Microsoft.NET.Runtime.MonoTargets.Sdk version 6.0.12... Writing workload pack installation record for Microsoft.NET.Runtime.MonoTargets.Sdk.net6 version 6.0.12... Garbage collecting for SDK feature band(s) 7.0.100... Successfully installed workload(s) maui-ios. ```

But looking at this output, it looks like it's installing a different version of Microsoft.iOS.Sdk? So I'm not sure if building my project would even use my locally built version.

ls -1 /usr/local/share/dotnet/packs/Microsoft.iOS.Sdk/

16.1.1483-ci.test
16.2.1004
16.2.3

goelze commented 1 year ago

@rolfbjarne I nuked my dotnet folder and nuget package cache and was able to install everything successfully (including the maui-ios workload). 😄 When archiving my app, is there a way to verify that it is using my locally built SDK?

Edit: was able to inspect the DLL files within the bundled application. I will test out this fix with our beta testers over the next week or so, and report back here.

rolfbjarne commented 1 year ago

@rolfbjarne I nuked my dotnet folder and nuget package cache and was able to install everything successfully (including the maui-ios workload). 😄 When archiving my app, is there a way to verify that it is using my locally built SDK?

Edit: was able to inspect the DLL files within the bundled application. I will test out this fix with our beta testers over the next week or so, and report back here.

That's great!

FWIW you can also look at the Info.plist, we add a com.microsoft.<platform> key with the version info used to build the app.

Alternatively you can also get a binary build log and check the path to the Microsoft.iOS.dll used in the build (which is really just a different version of what you did, so inspecting the assemblies post-build is equivalent).

narciszait commented 1 year ago

@rolfbjarne sorry to bother you again and change my mind, but the link for the package for iOS 15 does not work anymore. Any chance it can get "revived"? Our build machine is still on Xamarin.iOS 15 - i was a bit too fast to ask about an iOS 16 package

rolfbjarne commented 1 year ago

@narciszait can you get the exact version information from your build machines?

You can get the version information from the command line using the command:

/Library/Frameworks/Xamarin.iOS.framework/Versions/Current/bin/mtouch --version

narciszait commented 1 year ago

@rolfbjarne i get the following: mtouch 15.10.0.5 (d17-2: 96b3edb6d)

rolfbjarne commented 1 year ago

@narciszait I'm working on getting a build with the fix on top of that version, I'll let you know when I have the package.

goelze commented 1 year ago

I'm still seeing an error:

System.ObjectDisposedException: CancellationTokenSource_Disposed
  ?, in void CancellationTokenSource.ThrowIfDisposed()
  ?, in void CancellationTokenSource.Cancel(bool) x 2
  ?, in void NSUrlSessionHandlerDelegate.DidCompleteWithError(NSUrlSession, NSUrlSessionTask, NSError)

We're using sentry.io to collect error data, which seems like it's having trouble getting debug info in .NET 7 for iOS, so that's why the stack trace doesn't have a lot of detail.

Here's the native stack trace I pulled from the Xcode organizer

Thread 4 name:
Thread 4 name:
Thread 4 Crashed:
0   libsystem_kernel.dylib          0x0000000215370200 __pthread_kill + 8 (:-1)
1   libsystem_pthread.dylib         0x00000002257d21ac pthread_kill + 268 (pthread.c:1670)
2   libsystem_c.dylib               0x00000001dfda9c8c abort + 180 (abort.c:118)
3   F5Sports.App.iOS                0x0000000102876900 sigabrt_signal_handler.cold.1 + 48 (mini-posix.c:226)
4   F5Sports.App.iOS                0x000000010280d760 sigabrt_signal_handler + 168 (mini-posix.c:224)
5   libsystem_platform.dylib        0x0000000225737a90 _sigtramp + 56 (sigtramp.c:116)
6   libsystem_pthread.dylib         0x00000002257d21ac pthread_kill + 268 (pthread.c:1670)
7   libsystem_c.dylib               0x00000001dfda9c8c abort + 180 (abort.c:118)
8   F5Sports.App.iOS                0x00000001025bad3c xamarin_unhandled_exception_handler + 60 (runtime.m:1121)
9   F5Sports.App.iOS                0x00000001026f7c04 mono_invoke_unhandled_exception_hook + 132 (exception.c:1219)
10  F5Sports.App.iOS                0x00000001027f90a0 mono_handle_exception_internal + 6808 (mini-exceptions.c:2326)
11  F5Sports.App.iOS                0x00000001027f75fc mono_handle_exception + 52 (mini-exceptions.c:2668)
12  F5Sports.App.iOS                0x000000010280bb90 mono_arm_throw_exception + 360 (exceptions-arm64.c:402)
13  F5Sports.App.iOS                0x000000010259c30c throw_exception + 172
14  F5Sports.App.iOS                0x000000010273e92c mono_raise_exception_deprecated + 16 (object.c:7409)
15  F5Sports.App.iOS                0x000000010273e92c mono_raise_exception_internal + 32 (object.c:7386)
16  F5Sports.App.iOS                0x000000010273e92c mono_raise_exception + 56 (object.c:7398)
17  F5Sports.App.iOS                0x00000001025ba904 xamarin_process_managed_exception + 252 (runtime.m:2367)
18  F5Sports.App.iOS                0x0000000102858688 native_to_managed_trampoline_24(objc_object*, objc_selector*, _MonoMethod**, objc_object*, objc_object*, objc_object*, unsigned int) + 924 (registrar.mm:1465)
19  F5Sports.App.iOS                0x0000000102870960 -[System_Net_Http_NSUrlSessionHandler_NSUrlSessionHandlerDelegate URLSession:task:didCompleteWithError:] + 72 (registrar.mm:17200)
20  CFNetwork                       0x00000001d9907c0c __79-[__NSCFURLSessionDelegateWrapper task:didCompleteWithError:completionHandler:]_block_invoke + 36 (SessionDelegateWrapper.mm:448)
21  libdispatch.dylib               0x00000001dfd444b4 _dispatch_call_block_and_release + 32 (init.c:1518)
22  libdispatch.dylib               0x00000001dfd45fdc _dispatch_client_callout + 20 (object.m:560)
23  libdispatch.dylib               0x00000001dfd4d694 _dispatch_lane_serial_drain + 672 (inline_internal.h:2632)
24  libdispatch.dylib               0x00000001dfd4e214 _dispatch_lane_invoke + 436 (queue.c:3940)
25  libdispatch.dylib               0x00000001dfd58e10 _dispatch_workloop_worker_thread + 652 (queue.c:6846)
26  libsystem_pthread.dylib         0x00000002257cbdf8 _pthread_wqthread + 288 (pthread.c:2618)
27  libsystem_pthread.dylib         0x00000002257cbb98 start_wqthread + 8 (:-1)

rolfbjarne commented 1 year ago

@mandel-macaque ^

hjharvey-MSFT commented 1 year ago

@rolfbjarne @mandel-macaque Does anyone have any updates? From Microsoft side it would be nice to know if @goelze issue is because the fix doesn't work in .Net 7 or if it is something else.

@narciszait do you have a status update on how this is working on your end?

narciszait commented 1 year ago

@hjharvey-MSFT i cannot give you an update cause i am waiting for Xamarin.iOS 15 installer package, which is not available for download anymore @rolfbjarne mentioned he was building something on top of Xamarin.iOS 15, but haven't anything

rolfbjarne commented 1 year ago

@rolfbjarne mentioned he was building something on top of Xamarin.iOS 15, but haven't anything

We've had some CI trouble, which was more complicated for this particular case, because it's an older branch, but I'm still working on it and hopefully I'll have packages soon.

rolfbjarne commented 1 year ago

@rolfbjarne mentioned he was building something on top of Xamarin.iOS 15, but haven't anything

We've had some CI trouble, which was more complicated for this particular case, because it's an older branch, but I'm still working on it and hopefully I'll have packages soon.

It took me a while, but here's a link:

xamarin.ios-15.10.0.26.pkg (it's unsigned, so you'll have to right-click and select Open after downloading it instead of double-clicking it)

Just for reference, here's the link with the fix on top of the upcoming release for Xcode 14.2:

xamarin.ios-16.2.0.9.pkg

xamarin / xamarin-macios