search

tmenier / Flurl

Fluent URL builder and testable HTTP client for .NET
https://flurl.dev
MIT License
4.23k stars 387 forks source link

Inconsistent Cookies between Android and iOS in .NET MAUI #760

Open aaron-yb opened 1 year ago

aaron-yb commented 1 year ago

I have been debugging a package that implements Flurl. (Original issue here: https://github.com/bgmulinari/B1SLayer/issues/49). Have tried adjusting iOS info.plist file with NSAppTransportSecurity values.

On .NET Maui Android the following code will return session cookies:

` private async Task FlurlLoginAsync() { var loginResponse = await _serviceLayerUrl .AppendPathSegment("Login") .WithCookies(out var cookieJar) .PostJsonAsync(new { CompanyDB = _companyDB, UserName = _username, Password = _password }) .ReceiveString();

    Console.WriteLine("Cookies received:");
    cookieJar.ToList().ForEach(cookie => Console.WriteLine($"{cookie.Name}: {cookie.Value}"));
}

`

However when running the same from iOS, there are never any cookies. Debug output looks as following:

On iOS / Mac I get the following output:

2023-08-23 14:56:09.536898+0100 SLTest[10595:1499318] Request URL: https://url:50000/b1s/v1/Login 2023-08-23 14:56:09.537186+0100 SLTest[10595:1499318] HTTP Method: POST 2023-08-23 14:56:09.537798+0100 SLTest[10595:1499318] Response Status Code: OK 2023-08-23 14:56:09.537904+0100 SLTest[10595:1499318] Request Headers: 2023-08-23 14:56:09.538140+0100 SLTest[10595:1499318] Response Headers: 2023-08-23 14:56:09.538552+0100 SLTest[10595:1499318] Server: Apache 2023-08-23 14:56:09.538721+0100 SLTest[10595:1499318] Vary: Accept-Encoding 2023-08-23 14:56:09.540703+0100 SLTest[10595:1499318] Date: Wed, 23 Aug 2023 13:56:06 GMT 2023-08-23 14:56:09.540808+0100 SLTest[10595:1499318] Keep-Alive: timeout=5, max=100 2023-08-23 14:56:09.540898+0100 SLTest[10595:1499318] Connection: Keep-Alive 2023-08-23 14:56:09.542724+0100 SLTest[10595:1499318] Request Body: { "odata.metadata" : "https://url:50000/b1s/v1/$metadata#B1Sessions/@Element", "SessionId" : "ce6f595c-41bc-11ee-c000-000c2980395a-14548-6632", "Version" : "1000210", "SessionTimeout" : 30 } 2023-08-23 14:56:09.558334+0100 SLTest[10595:1498480] Cookies received:

On Android I get the following:

[DOTNET] Request URL: https://url:50000/b1s/v1/Login [DOTNET] HTTP Method: POST [DOTNET] Response Status Code: OK [DOTNET] Request Headers: [DOTNET] Response Headers: [DOTNET] Connection: Keep-Alive [DOTNET] Date: Wed, 23 Aug 2023 14:02:19 GMT [DOTNET] Keep-Alive: timeout=5, max=100 [DOTNET] Server: Apache [DOTNET] Set-Cookie: B1SESSION=ac58e080-41bd-11ee-c000-000c2980395a-14752-8204;HttpOnly;;Secure;SameSite=None, ROUTEID=.node8; path=/;Secure;SameSite=None [DOTNET] Vary: Accept-Encoding [DOTNET] X-Android-Received-Millis: 1692799340723 [DOTNET] X-Android-Response-Source: NETWORK 200 [DOTNET] X-Android-Selected-Protocol: http/1.1 [DOTNET] X-Android-Sent-Millis: 1692799338028 [DOTNET] Request Body: [DOTNET] { [DOTNET] "odata.metadata" : "https://url:50000/b1s/v1/$metadata#B1Sessions/@Element", [DOTNET] "SessionId" : "ac58e080-41bd-11ee-c000-000c2980395a-14752-8204", [DOTNET] "Version" : "1000210", [DOTNET] "SessionTimeout" : 30 [DOTNET] } [DOTNET] Cookies received: [DOTNET] ROUTEID: .node8 [DOTNET] B1SESSION: ac58e080-41bd-11ee-c000-000c2980395a-14752-8204 [DOTNET] Request URL: https://url:50000/b1s/v1/Login [DOTNET] HTTP Method: POST [DOTNET] Response Status Code: OK [DOTNET] Request Headers: [DOTNET] Response Headers: [DOTNET] Connection: Keep-Alive [DOTNET] Date: Wed, 23 Aug 2023 14:02:19 GMT [DOTNET] Keep-Alive: timeout=5, max=100 [DOTNET] Server: Apache [DOTNET] Set-Cookie: B1SESSION=ac567d90-41bd-11ee-c000-000c2980395a-8440-7940;HttpOnly;;Secure;SameSite=None, ROUTEID=.node7; path=/;Secure;SameSite=None [DOTNET] Vary: Accept-Encoding [DOTNET] X-Android-Received-Millis: 1692799340837 [DOTNET] X-Android-Response-Source: NETWORK 200 [DOTNET] X-Android-Selected-Protocol: http/1.1 [DOTNET] X-Android-Sent-Millis: 1692799338028 [DOTNET] Request Body: [DOTNET] { [DOTNET] "odata.metadata" : "https://url:50000/b1s/v1/$metadata#B1Sessions/@Element", [DOTNET] "SessionId" : "ac567d90-41bd-11ee-c000-000c2980395a-8440-7940", [DOTNET] "Version" : "1000210", [DOTNET] "SessionTimeout" : 30 [DOTNET] } [DOTNET] Cookies received: [DOTNET] ROUTEID: .node7 [DOTNET] B1SESSION: ac567d90-41bd-11ee-c000-000c2980395a-8440-7940

bgmulinari commented 1 year ago

As tested here by @aaron-yb, when using just the native HttpClient, the Set-Cookie header is present in the response, but not present when performing the same request using Flurl.

This results in an empty CookieJar when calling WithCookies with an output parameter.

tmenier commented 1 year ago

Hmm. I'm not set up with MAUI at the moment, but @aaron-yb if you could help me troubleshoot this I would appreciate it. Please try running this on iOS, based on your code sample above:

var handler = new HttpClientHandler();
handler.UserCookies = false;
handler.ServerCertificateCustomValidationCallback = (sender, cert, chain, sslPolicyErrors) => true;
var httpCli = new HttpClient(handler);
var flurlCli = new FlurlClient(httpCli);

var flurlResp = await flurlCli
  .Request(_serviceLayerUrl)
  .AppendPathSegment("Login")
  .WithCookies(out var cookieJar)
  .PostJsonAsync(new { CompanyDB = _companyDB, UserName = _username, Password = _password });

// what's in cookieJar?
// what's in flurlResp.Cookies?
// what's in flurlResp.Headers?

For the comments at the bottom, you could either output that info to the console or just add a breakpoint and let me know what you're seeing. I think this will be helpful in getting to the bottom of the problem. Thanks!

aaron-yb commented 1 year ago

Hi @tmenier , apologies for the delay in getting back to you.

I get the following output on iOS:

2023-09-29 15:54:05.473476+0100 SLTest[5435:417741] cookieJar: 2023-09-29 15:54:05.473875+0100 SLTest[5435:417741] flurlResp.Cookies: 2023-09-29 15:54:05.473998+0100 SLTest[5435:417741] flurlResp.Headers 2023-09-29 15:54:05.477067+0100 SLTest[5435:417741] Server: Apache 2023-09-29 15:54:05.477190+0100 SLTest[5435:417741] Vary: Accept-Encoding 2023-09-29 15:54:05.477312+0100 SLTest[5435:417741] Date: Fri, 29 Sep 2023 14:54:00 GMT 2023-09-29 15:54:05.477409+0100 SLTest[5435:417741] Keep-Alive: timeout=5, max=100 2023-09-29 15:54:05.477496+0100 SLTest[5435:417741] Connection: Keep-Alive 2023-09-29 15:54:05.477588+0100 SLTest[5435:417741] Content-Type: application/json; odata=minimalmetadata; charset=utf-8 2023-09-29 15:54:05.477670+0100 SLTest[5435:417741] Content-Encoding: gzip 2023-09-29 15:54:05.477745+0100 SLTest[5435:417741] Content-Length: 181 2023-09-29 15:54:05.477818+0100 SLTest[5435:417741] Keep-Alive: timeout=5, max=100

and the following on Android:

[DOTNET] cookieJar: [DOTNET] ROUTEID: .node6 [DOTNET] B1SESSION: ba7e3f4c-5ed8-11ee-c000-000c2980395a-8972-7672 [DOTNET] flurlResp.Cookies: [DOTNET] B1SESSION: ba7e3f4c-5ed8-11ee-c000-000c2980395a-8972-7672 [DOTNET] ROUTEID: .node6 [DOTNET] flurlResp.Headers [DOTNET] Connection: Keep-Alive [DOTNET] Date: Fri, 29 Sep 2023 14:59:02 GMT [DOTNET] Keep-Alive: timeout=5, max=100 [DOTNET] Server: Apache [DOTNET] Set-Cookie: B1SESSION=ba7e3f4c-5ed8-11ee-c000-000c2980395a-8972-7672;HttpOnly;;Secure;SameSite=None [DOTNET] Set-Cookie: ROUTEID=.node6; path=/;Secure;SameSite=None [DOTNET] Transfer-Encoding: chunked [DOTNET] Vary: Accept-Encoding [DOTNET] X-Android-Received-Millis: 1695999546241 [DOTNET] X-Android-Response-Source: NETWORK 200 [DOTNET] X-Android-Selected-Protocol: http/1.1 [DOTNET] X-Android-Sent-Millis: 1695999542335 [DOTNET] Content-Type: application/json; odata=minimalmetadata; charset=utf-8

tmenier commented 1 year ago

Interesting. Normally, when HttpClientHandler.UseCookies is true (the default), Set-Cookie headers are intercepted by the handler and never come through in HttpRequestMessage.Headers. Flurl needs to see those headers in order for CookieJar to work, so it sets that value to false. But apparently those Set-Cookie headers still aren't coming through.

However, what I just noticed looking back at this snippet from @bgmulinari, UseCookies is never turned off, yet apparently those headers are coming through.

It's as though UseCookies is behaving exactly the opposite on iOS as on other platforms!

To test this theory, could you run my snippet above yet again, only remove this line entirely?

handler.UseCookies = false;
aaron-yb commented 1 year ago

Hi @tmenier ,

Looks like you may be on to something!

Here is the output on iOS:

2023-09-30 14:59:15.240374+0100 SLTest[9230:629017] cookieJar: 2023-09-30 14:59:15.240711+0100 SLTest[9230:629017] ROUTEID: .node1 2023-09-30 14:59:15.240812+0100 SLTest[9230:629017] B1SESSION: 883a27aa-5f99-11ee-c000-000c2980395a-7456-7780 2023-09-30 14:59:15.240941+0100 SLTest[9230:629017] flurlResp.Cookies: 2023-09-30 14:59:15.241125+0100 SLTest[9230:629017] B1SESSION: 883a27aa-5f99-11ee-c000-000c2980395a-7456-7780 2023-09-30 14:59:15.241209+0100 SLTest[9230:629017] ROUTEID: .node1 2023-09-30 14:59:15.241285+0100 SLTest[9230:629017] flurlResp.Headers 2023-09-30 14:59:15.244630+0100 SLTest[9230:629017] Server: Apache 2023-09-30 14:59:15.244732+0100 SLTest[9230:629017] Vary: Accept-Encoding 2023-09-30 14:59:15.244825+0100 SLTest[9230:629017] Date: Sat, 30 Sep 2023 13:59:11 GMT 2023-09-30 14:59:15.244914+0100 SLTest[9230:629017] Keep-Alive: timeout=5, max=100 2023-09-30 14:59:15.245062+0100 SLTest[9230:629017] Connection: Keep-Alive 2023-09-30 14:59:15.245135+0100 SLTest[9230:629017] Set-Cookie: B1SESSION=883a27aa-5f99-11ee-c000-000c2980395a-7456-7780; Path=/b1s/v1; Domain= xxx.xxx.xxx; Version=0; Discard; Secure; httponly 2023-09-30 14:59:15.245234+0100 SLTest[9230:629017] Set-Cookie: ROUTEID=.node1; Path=/; Domain=xxx.xxx.xxx; Version=0; Discard; Secure 2023-09-30 14:59:15.245298+0100 SLTest[9230:629017] Content-Type: application/json; odata=minimalmetadata; charset=utf-8 2023-09-30 14:59:15.245366+0100 SLTest[9230:629017] Content-Encoding: gzip 2023-09-30 14:59:15.245423+0100 SLTest[9230:629017] Content-Length: 181 2023-09-30 14:59:15.245486+0100 SLTest[9230:629017] Keep-Alive: timeout=5, max=100

(domain outputs normally, but I have obscured it)

Here is the output on Android:

[DOTNET] ROUTEID: .node9 [DOTNET] B1SESSION: 19955720-5f99-11ee-c000-000c2980395a-6148-7472 [DOTNET] flurlResp.Cookies: [DOTNET] B1SESSION: 19955720-5f99-11ee-c000-000c2980395a-6148-7472 [DOTNET] ROUTEID: .node9 [DOTNET] flurlResp.Headers [DOTNET] Connection: Keep-Alive [DOTNET] Date: Sat, 30 Sep 2023 13:56:05 GMT [DOTNET] Keep-Alive: timeout=5, max=100 [DOTNET] Server: Apache [DOTNET] Set-Cookie: B1SESSION=19955720-5f99-11ee-c000-000c2980395a-6148-7472;HttpOnly;;Secure;SameSite=None [DOTNET] Set-Cookie: ROUTEID=.node9; path=/;Secure;SameSite=None [DOTNET] Transfer-Encoding: chunked [DOTNET] Vary: Accept-Encoding [DOTNET] X-Android-Received-Millis: 1696082170869 [DOTNET] X-Android-Response-Source: NETWORK 200 [DOTNET] X-Android-Selected-Protocol: http/1.1 [DOTNET] X-Android-Sent-Millis: 1696082165165 [DOTNET] Content-Type: application/json; odata=minimalmetadata; charset=utf-8

tmenier commented 1 year ago

Wow. I think you've proven that UseCookies behaves exactly the opposite on iOS than it does on other platforms. I'm going the leave the help-wanted label on here in hopes that you (or someone else) can take the next steps:

  1. Figure out which .NET repository holds this quirky implementation. Open an issue with that team, confirm whether it is bug, if/when they anticipate a fix, and/or what their recommendation is for a work-around.
  2. If I must resort to conditional compilation, which preprocessor symbol(s) should be used for the work-around?

Armed with that information, I'll be willing to move forward with a fix. Thanks!

tmenier commented 1 year ago

Here's the default handler used on iOS platforms: https://learn.microsoft.com/en-us/dotnet/api/system.net.http.nsurlsessionhandler

And I believe this is the repo where an issue should be raised: https://github.com/xamarin/xamarin-macios

I might have time to pursue this in the near future, but if someone wants to beat me to it, by all means. :)

aaron-yb commented 1 year ago

Hey @tmenier thanks for your replies and info- I'm having a go at working this all out, bit of a learning curve! Will see who gets there first