Sending image over http crashes iOS app, Release builds only

DanielGlick commented 3 years ago

Steps to Reproduce

byte[] testSignatureBytes = new byte[]
            {
0x42, 0x4d, 0x06, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x76, 0x00, 0x00, 0x00, 0x28, 0x00,
  0x00, 0x00, 0x19, 0x00, 0x00, 0x00, 0x19, 0x00, 0x00, 0x00, 0x01, 0x00, 0x04, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x90, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x80, 0x00, 0x00, 0x80,
  0x00, 0x00, 0x00, 0x80, 0x80, 0x00, 0x80, 0x00, 0x00, 0x00, 0x80, 0x00, 0x80, 0x00, 0x80, 0x80,
  0x00, 0x00, 0x80, 0x80, 0x80, 0x00, 0xc0, 0xc0, 0xc0, 0x00, 0x00, 0x00, 0xff, 0x00, 0x00, 0xff,
  0x00, 0x00, 0x00, 0xff, 0xff, 0x00, 0xff, 0x00, 0x00, 0x00, 0xff, 0x00, 0xff, 0x00, 0xff, 0xff,
  0x00, 0x00, 0xff, 0xff, 0xff, 0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x70, 0x07, 0xff, 0xff,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0x70, 0x00, 0x00, 0x7f, 0xff,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0x70, 0x00, 0x00, 0x00, 0x07, 0xff,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0x70, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7f,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xf7, 0x00, 0x00, 0x00, 0x7f, 0x80, 0x00, 0x07,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xf0, 0x00, 0x00, 0x7f, 0xff, 0xf8, 0x00, 0x00,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0x70, 0x00, 0x7f, 0xff, 0xff, 0xff, 0x80, 0x00,
  0x7f, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0xf0, 0x00,
  0x0f, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xf7, 0x00, 0x07, 0xff, 0xff, 0xff, 0xff, 0xf8, 0x00,
  0x0f, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xf0, 0x00, 0x0f, 0xff, 0xff, 0xff, 0xff, 0xf7, 0x00,
  0x0f, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xf0, 0x00, 0x7f, 0xff, 0xff, 0xff, 0xff, 0xf0, 0x00,
  0x0f, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xf8, 0x07, 0xff, 0xff, 0xff, 0xff, 0xff, 0xf0, 0x00,
  0x7f, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xf8, 0x07,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0x70, 0x00, 0x7f, 0x00, 0x00, 0x7f,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0x00, 0x00, 0x0f, 0x70, 0x00, 0x0f,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0x00, 0x00, 0x0f, 0xf0, 0x00, 0x7f,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0x80, 0x00, 0x7f, 0xf8, 0x07, 0xff,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00
            };
var stream = new MemoryStream(testSignatureBytes, 0, testSignatureBytes.Length, true, true);

var requestMessage = new HttpRequestMessage(HttpMethod.POST, [request url])
requestMessage.Content = new StreamContent(stream);
requestMessage.Content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
requestMessage.Headers.TransferEncodingChunked = true;

 var responseMessage = await httpClient.SendAsync(requestMessage,HttpCompletionOption.ResponseHeadersRead).ConfigureAwait(false);

Expected Behavior

Should send the request containing the stream to the server

Actual Behavior

App crashes from a native exception...

SIGABRT: Objective-C exception thrown. Name: NSInvalidArgumentException Reason: *** -streamStatus only defined for abstract class. Define -[System_Net_Http_NSUrlSessionHandler_WrappedNSInputStream streamStatus]

I can verify that this is not an issue on the servers end. I put a breakpoint on the POST Route and it was never hit.

Environment

=== Visual Studio Community 2019 for Mac ===

Version 8.9.9 (build 3)
Installation UUID: ffc52ff2-88e8-4710-a639-2d714fb8ad47
    GTK+ 2.24.23 (Raleigh theme)
    Xamarin.Mac 6.18.0.23 (d16-6 / 088c73638)

    Package version: 612000125

=== Mono Framework MDK ===

Runtime:
    Mono 6.12.0.125 (2020-02/8c552e98bd6) (64-bit)
    Package version: 612000125

=== Roslyn (Language Service) ===

3.9.0-6.21152.10+c10f884b30737542ddd84ca889a4aad9281ce210

=== NuGet ===

Version: 5.8.0.6860

=== .NET Core SDK ===

SDK: /usr/local/share/dotnet/sdk/5.0.203/Sdks
SDK Versions:
    5.0.203
    5.0.201
    5.0.101
    5.0.100
    3.1.409
    3.1.407
    3.1.404
    3.1.403
    3.1.402
    3.1.401
    3.1.302
    3.1.301
    3.1.300
    3.1.200
    3.1.102
    3.1.101
    3.1.100
    3.0.101
    3.0.100
    2.2.300
    2.1.700
    2.1.505
    2.1.402
    2.1.302
    2.1.301
    2.1.4
    2.0.0
MSBuild SDKs: /Applications/Visual Studio.app/Contents/Resources/lib/monodevelop/bin/MSBuild/Current/bin/Sdks

=== .NET Core Runtime ===

Runtime: /usr/local/share/dotnet/dotnet
Runtime Versions:
    5.0.6
    5.0.4
    5.0.1
    5.0.0
    3.1.15
    3.1.13
    3.1.10
    3.1.9
    3.1.8
    3.1.7
    3.1.6
    3.1.5
    3.1.4
    3.1.2
    3.1.1
    3.1.0
    3.0.1
    3.0.0
    2.2.5
    2.1.23
    2.1.22
    2.1.21
    2.1.20
    2.1.19
    2.1.18
    2.1.16
    2.1.15
    2.1.14
    2.1.13
    2.1.11
    2.1.9
    2.1.4
    2.1.2
    2.1.1
    2.0.5
    2.0.0

=== .NET Core 3.1 SDK ===

SDK: 3.1.409

=== Xamarin.Profiler ===

Version: 1.6.12.26
Location: /Applications/Xamarin Profiler.app/Contents/MacOS/Xamarin Profiler

=== Updater ===

Version: 11

=== Xamarin.Android ===

Version: 11.2.2.1 (Visual Studio Community)
Commit: xamarin-android/d16-9/877f572
Android SDK: /Users/ecc/Library/Developer/Xamarin/android-sdk-macosx
    Supported Android versions:
        8.1 (API level 27)

SDK Tools Version: 26.1.1
SDK Platform Tools Version: 30.0.5
SDK Build Tools Version: 29.0.2

Build Information: 
Mono: 5e9cb6d
Java.Interop: xamarin/java.interop/d16-9@54f8c24
ProGuard: Guardsquare/proguard/v7.0.1@912d149
SQLite: xamarin/sqlite/3.34.1@daff8f4
Xamarin.Android Tools: xamarin/xamarin-android-tools/d16-9@d210f11

=== Microsoft OpenJDK for Mobile ===

Java SDK: /Users/ecc/Library/Developer/Xamarin/jdk/microsoft_dist_openjdk_1.8.0.25
1.8.0-25
Android Designer EPL code available here:
https://github.com/xamarin/AndroidDesigner.EPL

=== Android SDK Manager ===

Version: 16.9.0.22
Hash: a391de2
Branch: remotes/origin/dev/bretjohn/readme-tweak~3
Build date: 2021-04-21 21:23:40 UTC

=== Android Device Manager ===

Version: 16.9.0.17
Hash: fc2b3db
Branch: remotes/origin/dev/jmt/d16-9bump~1
Build date: 2021-04-21 21:23:58 UTC

=== Apple Developer Tools ===

Xcode 12.5 (18205)
Build 12E262

=== Xamarin.Mac ===

Version: 7.10.0.5 (Visual Studio Community)
Hash: ab40b131d
Branch: xcode12.5
Build date: 2021-04-22 08:56:41-0400

=== Xamarin.iOS ===

Version: 14.16.0.5 (Visual Studio Community)
Hash: ab40b131d
Branch: xcode12.5
Build date: 2021-04-22 08:56:42-0400

=== Xamarin Designer ===

Version: 16.9.0.324
Hash: b1e216c75
Branch: remotes/origin/d16-9
Build date: 2021-04-16 00:02:50 UTC

=== Build Information ===

Release ID: 809090003
Git revision: 163d834aa64ba9129e5ef30e49a947ef67bcd809
Build date: 2021-05-06 10:14:43-04
Build branch: release-8.9
Xamarin extensions: 163d834aa64ba9129e5ef30e49a947ef67bcd809

=== Operating System ===

Mac OS X 10.16.0
Darwin 20.4.0 Darwin Kernel Version 20.4.0
    Thu Apr 22 21:46:47 PDT 2021
    root:xnu-7195.101.2~1/RELEASE_X86_64 x86_64

DanielGlick commented 3 years ago

Posting this here for help. I have not seen any reports of similar issues, and I cannot find any solutions to fix this issue.

rolfbjarne commented 3 years ago

The following code works just fine for me:

        byte [] testSignatureBytes = new byte []
           {
0x42, 0x4d, 0x06, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x76, 0x00, 0x00, 0x00, 0x28, 0x00,
  0x00, 0x00, 0x19, 0x00, 0x00, 0x00, 0x19, 0x00, 0x00, 0x00, 0x01, 0x00, 0x04, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x90, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x80, 0x00, 0x00, 0x80,
  0x00, 0x00, 0x00, 0x80, 0x80, 0x00, 0x80, 0x00, 0x00, 0x00, 0x80, 0x00, 0x80, 0x00, 0x80, 0x80,
  0x00, 0x00, 0x80, 0x80, 0x80, 0x00, 0xc0, 0xc0, 0xc0, 0x00, 0x00, 0x00, 0xff, 0x00, 0x00, 0xff,
  0x00, 0x00, 0x00, 0xff, 0xff, 0x00, 0xff, 0x00, 0x00, 0x00, 0xff, 0x00, 0xff, 0x00, 0xff, 0xff,
  0x00, 0x00, 0xff, 0xff, 0xff, 0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x70, 0x07, 0xff, 0xff,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0x70, 0x00, 0x00, 0x7f, 0xff,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0x70, 0x00, 0x00, 0x00, 0x07, 0xff,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0x70, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7f,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xf7, 0x00, 0x00, 0x00, 0x7f, 0x80, 0x00, 0x07,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xf0, 0x00, 0x00, 0x7f, 0xff, 0xf8, 0x00, 0x00,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0x70, 0x00, 0x7f, 0xff, 0xff, 0xff, 0x80, 0x00,
  0x7f, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0xf0, 0x00,
  0x0f, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xf7, 0x00, 0x07, 0xff, 0xff, 0xff, 0xff, 0xf8, 0x00,
  0x0f, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xf0, 0x00, 0x0f, 0xff, 0xff, 0xff, 0xff, 0xf7, 0x00,
  0x0f, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xf0, 0x00, 0x7f, 0xff, 0xff, 0xff, 0xff, 0xf0, 0x00,
  0x0f, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xf8, 0x07, 0xff, 0xff, 0xff, 0xff, 0xff, 0xf0, 0x00,
  0x7f, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xf8, 0x07,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0x70, 0x00, 0x7f, 0x00, 0x00, 0x7f,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0x00, 0x00, 0x0f, 0x70, 0x00, 0x0f,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0x00, 0x00, 0x0f, 0xf0, 0x00, 0x7f,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0x80, 0x00, 0x7f, 0xf8, 0x07, 0xff,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xf0, 0x00, 0x00, 0x00
           };
        var stream = new MemoryStream (testSignatureBytes, 0, testSignatureBytes.Length, true, true);

        var requestMessage = new HttpRequestMessage (HttpMethod.Post, "https://httpbin.org/post"); 
        requestMessage.Content = new StreamContent (stream);
        requestMessage.Content.Headers.ContentType = new MediaTypeHeaderValue ("application/octet-stream");
        requestMessage.Headers.TransferEncodingChunked = true;

        var httpClient = new HttpClient ();
        Task.Run (async () => {
            var responseMessage = await httpClient.SendAsync (requestMessage, HttpCompletionOption.ResponseHeadersRead).ConfigureAwait (false);
            Console.WriteLine (responseMessage);
        });

so please attach a complete test project we can use to reproduce this.

DanielGlick commented 3 years ago

I cannot attach the app I am using to reproduce this. I will try to reproduce in an app that I can attach here. Can you spot anything from the crash error attached that could point to a problem?

rolfbjarne commented 3 years ago

Can you spot anything from the crash error attached that could point to a problem?

My guess is that it's server-dependent somehow, even though server-side breakpoint was never hit (https related? auth related? basically any kind of communication that happens between the client and the server before your server-side code is executed).

DanielGlick commented 3 years ago

@rolfbjarne I am curious how reproduction projects are usually made. This is a unique situation because I am only reproducing it in a Release build of my app. I could create a demo application but I think to run it in release I need to have it registered with Apple correct?

DanielGlick commented 3 years ago

Okay I have a new development here. I turned off linking in our application (previously was set to Framework SDK's only), and that seems to have fixed the issue. One other thing I should have mentioned is that this is a Xamarin Forms application.

Also, since it seems to be a linker problem, I turned on linking in Debug and that reproduces the crash.

DanielGlick commented 3 years ago

I also noticed https://github.com/xamarin/xamarin-macios/blob/6898920c72af2bd1c6722b7cb849da2423702220/src/Foundation/NSUrlSessionHandler.cs#L1188

That comment seems to describe a very similar issue as what I am seeing.

rolfbjarne commented 3 years ago

@rolfbjarne I am curious how reproduction projects are usually made.

Just create a new project from a template, change it so that it fails the same way, zip it up and attach it here.

DanielGlick commented 3 years ago

We were able to work around the issue by setting the httpclient of our app to managed vs NSUrlSession. Also turning on linking to framework skds only we were able to reproduce this in debug. I will try to get a reproduction project created so you can debug this in your end. Seems like it definitely is related to my comment above. @rolfbjarne Thoughts on my comment above in NSUrlSessionHandler.cs?

rolfbjarne commented 3 years ago

I will try to get a reproduction project created so you can debug this in your end. Seems like it definitely is related to my comment above.

That would be great!

Thoughts on my comment above in NSUrlSessionHandler.cs?

Yes, it could very well be related. However, until I can reproduce the issue myself, I can't know for sure.

mandel-macaque commented 3 years ago

If we have a repro project I can take a look at this issue. I am also curious (because I know the NSUrlSessionHandler well) if you have the same issue when the method is not using async.

spouliot commented 2 years ago

We have not got any reproducible test case to investigate the issue. If you can get one please open a new issue, attach the test case, and reference this (original issue). Thanks!

flimtix commented 2 years ago

I got the same error. When I take an image from the gallery and convert it to a base64 string everything works as intended. But as soon as I take a picture directly with the camera and send its Base64 string I get this error.

I use Http and the HttpClient to send the data.

Unfortunately I can't create a project which is the same as it is a productive system.... On the other side is a windows service which works with the netframework. This one receives my requests.

When I do the same on Android I don't get any errors. Also, I can enable NSAllowsArbitraryLoads which fixes the error more or less. But this is not a nice variant!

flimtix commented 2 years ago

I have found a possible cause in my case. 🥴 Most likely you set up an HttpClient like this or used the default timeout:

httpClient = new HttpClient {  Timeout = TimeSpan.FromSeconds(5)};

Xamarin has a lot of problems with the timeout from the HttpClient. Among other things, that the timeout is ignored when fetching data. When sending, this timeout is strangely respected. That means, as soon as your sending takes too long it will be aborted. This should be recorded as a bug and fixed... (Better error message / fix HttpClient)

xamarin / xamarin-macios