Native iOS/macOS HttpClientHandler incorrectly encodes query strings

abnegate commented 1 month ago

Steps to Reproduce

Instantiate HttpClient
Instantiate HttpRequestMessage for a GET request, with a request URI including a query string with special characters
Send the request using the client with the SendAsync method
Observe that during the SendAsync call, the query string is encoded incorrectly
Add <UseNativeHandler>false</UseNativeHandler> to project file
Observe that query string is not encoded incorrectly

Expected Behavior

Query string is encoded correctly

Actual Behavior

Query string is encoded incorrectly

Environment

Version information

``` .NET SDK: Version: 8.0.300 Commit: 326f6e68b2 Workload version: 8.0.300-manifests.4c099cbd MSBuild version: 17.10.4+10fbfbf2e Runtime Environment: OS Name: Mac OS X OS Version: 14.4 OS Platform: Darwin RID: osx-arm64 Base Path: /usr/local/share/dotnet/sdk/8.0.300/ .NET workloads installed: [maui] Installation Source: SDK 8.0.300 Manifest Version: 8.0.21/8.0.100 Manifest Path: /usr/local/share/dotnet/sdk-manifests/8.0.100/microsoft.net.sdk.maui/8.0.21/WorkloadManifest.json Install Type: FileBased Host: Version: 8.0.5 Architecture: arm64 Commit: 087e15321b .NET SDKs installed: 7.0.301 [/usr/local/share/dotnet/sdk] 6.0.400 [/usr/local/share/dotnet/sdk] 6.0.408 [/usr/local/share/dotnet/sdk] 6.0.412 [/usr/local/share/dotnet/sdk] 6.0.413 [/usr/local/share/dotnet/sdk] 8.0.306 [/usr/local/share/dotnet/sdk] 7.0.307 [/usr/local/share/dotnet/sdk] 9.0.201 [/usr/local/share/dotnet/sdk] 8.0.300 [/usr/local/share/dotnet/sdk] .NET runtimes installed: Microsoft.AspNetCore.App 6.0.6 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 6.0.8 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 6.0.16 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 6.0.20 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 6.0.21 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 7.0.9 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 7.0.10 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 8.0.2 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 8.0.5 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.NETCore.App 6.0.8 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 6.0.16 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 6.0.20 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 6.0.21 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 7.0.9 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 7.0.10 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 8.0.2 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 8.0.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Other architectures found: x64 [/usr/local/share/dotnet/x64] registered at [/etc/dotnet/install_location_x64] Environment variables: Not set global.json file: Not found Learn more: https://aka.ms/dotnet/info Download .NET: https://aka.ms/dotnet/download ```

Build Logs

msbuild.binlog.zip

Example Project (If Possible)

MauiApp1.zip

Before request sent

After request sent

rolfbjarne commented 1 month ago

I can reproduce the behavior.

The HttpResponse.RequestUri is changed/updated here:

https://github.com/xamarin/xamarin-macios/blob/e52af40a99b976289bad0dece6a30a5c8a83e815/src/Foundation/NSUrlSessionHandler.cs#L892

this code is rather old, but apparently RequestUri is updated to reflect any redirects:

https://github.com/xamarin/ModernHttpClient/commit/518ac1bf1921d1f02bcbbd5b895ff0fe74a2181f

rolfbjarne commented 1 month ago

Updating RequestUri after a redirect seems correct, a console app does that.

Test code:

async static Task Main ()
{
    var client = new HttpClient () { BaseAddress = new Uri("https://httpbin.org") };
    var request = new HttpRequestMessage (HttpMethod.Get,
        "/redirect-to?url=https%3A%2F%2Fmicrosoft.com&status_code=302&queries[]={}"
    );
    Console.WriteLine ($"Request uri 1: {request.RequestUri}");
    var response = await client.SendAsync(request);
    Console.WriteLine ($"Request uri 2: {request.RequestUri}");
    var content = await response.Content.ReadAsStringAsync();
    Console.WriteLine ((int) response.StatusCode);
    Console.WriteLine(content.Length);
}

prints:

Request uri 1: /redirect-to?url=https%3A%2F%2Fmicrosoft.com&status_code=302&queries[]={}
Request uri 2: https://www.microsoft.com/es-es/
200
201252

rolfbjarne commented 1 month ago

Looks like SocketsHttpHandler only updates RequestUri in case of a redirect:

https://github.com/dotnet/runtime/blob/f5eb26e4da0d0d7a5874553d8a793a551c34af0a/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/RedirectHandler.cs#L65-L66

xamarin / xamarin-macios