5.2.0 cookies logic breaks some SSO schemes

grablair commented 2 weeks ago

Expected Behavior

Necessary background: I work for Amazon. There are thousands of internal websites owned and maintained by thousands of different teams. There are a plethora of userscripts in use, with many different goals and purposes, and that have been in use for many years to create usecase-specific modifications to websites owned by other teams.

The way the XHR request cookie behavior works in v5.2.0 vs v5.1.1 is breaking many of the userscripts used internally due to the way our SSO auth process functions. All of our internal sites perform SSO auth in the following way:

Go to target URL
Service checks to see if there are SSO cookies present. If so, validates them and then determines authZ. If authorized, process request.
If the SSO cookies are not present, or if they are expired, then the target site responds with a set-cookie header, containing an SSO forgery protection token. It then redirects the user to our internal auth service.
The internal auth service authenticates the user (via credentials or session cookies), generates a token for the initial target URL, and then redirects the user back to the initial target URL with the token added to the request.
The initial site takes the token from the auth request and stores it in a cookie, and redirects it to the original URL, without the token in the request.
Return to step 2 (since the SSO tokens are now present and valid, the request is then processed)

Actual Behavior

Steps 1-3 function fine, but when redirected to the auth website, no cookies are added to the request (thus, no session cookie), and the auth website responds with a 401.

I can temporarily mitigate this by going to the target site manually, but our SSO tokens don't have a long liveness (5-10 mins) before they need to be refreshed, so it isn't a sustainable solution. Additionally, since we have hundreds of internal sites which each store their own SSO tokens, the user would have to visit every one that their scripts call, every 5-10 minutes.

I have tried to mess with different settings in the GM_xhr function's parameters, but to no avail. If I am missing some setting, please let me know! :)

There are surely other large companies whose employees use userscripts which also have similar SSO authN processes, which would also benefit from a simple solution to this problem.

Specifications

Chromium: 126.0.6478.56
TM: 5.2.0
OS: MacOS 14.4.1

Script

Sorry, the script relies on internal websites that are unreachable from the internet.

derjanb commented 2 weeks ago

Thanks for your bug report. In the meantime you can backup your scripts and use Tampermonkey Legacy which will stay MV2 as long as it is supported by the Chrome Webstore.

derjanb commented 2 weeks ago

@grablair I understand that you can't share your script, but can you please make this one fail?

Are your cookies secure, httpOnly, partitioned?

My test successfully sends the cookie to httpbin.org that was set by a GM.xmlHttpRequest before. Also the set-cookie header is present as expected.

// ==UserScript==
// @name         xxx
// @namespace    xxx
// @version      xxx
// @description  xxx
// @author       xxx
// @match        https://example.com
// @grant        GM.xmlHttpRequest
// @connect      httpbin.org
// ==/UserScript==

const d = Date.now();

const p = await GM.xmlHttpRequest({
    url: 'https://httpbin.org/response-headers?set-cookie=nonpartitioned=' + d + ';path=/;expires=Wed,%2021%20Sep%202033%2015:59:37%20GMT;httponly;secure;samesite=none'
    // url: 'https://httpbin.org/response-headers?set-cookie=nonpartitioned=' + d + ';path=/;samesite=none'
})
console.log(p.responseHeaders);

const r = await GM.xmlHttpRequest({ url: 'https://httpbin.org/cookies' });

console.log('Cookie Value should be ' + d);
console.log(r.responseText);

logs

grablair commented 2 weeks ago

Thanks for the quick response @derjanb! I'll take a look at your example code and let you know the result.

In the meantime I'll try to show clearly below exactly how each request-response is occurring and what cookies were present in the system cookies store before/after each request. Also, I tried this with the latest Beta release from the Chrome app store.

Initial state

Cookies for mysite.example.com cleared entirely
Cookies for auth.example.com stored in the system store include an authenticated session token. The session cookie is httpOnly, secure, samesite=none, and has no partition

Test 1: No SSO Cookies

Example code

GM_xmlhttpRequest({
    method: "GET",
    anonymous: false, // not sure if even necessary; tried with and without
    withCredentials: false, // not sure if even necessary; tried with and without
    url: `https://mysite.example.com/foo`,
    onload: myLoadHandler,
    onerror: myErrorHandler
});

Background requests

1. GET to mysite.example.com

Request headers

Accept: */*
Accept-Language: en-US,en;q=0.9
priority: u=1, i
sec-fetch-dest: empty
sec-fetch-mode: cors
sec-fetch-site: none
sec-gpc: 1

Response

HTTP/1.1 307 Temporary Redirect

Location: https://auth.example.com/SSO/redirect?redirect_uri=https://mysite.example.com
set-cookie: sso_rfp=123456789abcdef; Path=/; Mat-Age=36000; Secure; HttpOnly; SameSite=None

NOTE: set-cookie does get sent back, but the resulting cookie doesn't get stored.

2. GET to auth.example.com

Request headers

Accept: */*
Accept-Encoding: gzip, deflate, br, zstd
Accept-Language: en-US,en;q=0.9
Connection: keep-alive
Host: auth.example.com
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: none
Sec-GPC: 1
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36

NOTE: No session cookie presented, despite being in the system store for auth.example.com

Response

HTTP/1.1 401 Unauthorized

Server: Server
Date: Sat, 15 Jun 2024 19:24:40 GMT
Content-Type: */*; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Permitted-Cross-Domain-Policies: none
Referrer-Policy: strict-origin-when-cross-origin
Vary: Accept
Cache-Control: no-cache
X-Request-Id: b8b538dd-3b1e-437a-bb8c-81cae07f83a1
Strict-Transport-Security: max-age=63072000; includeSubDomains

New State

No change in system-stored cookies at all.

Still no cookies for mysite.example.com
Cookies for auth.example.com stored in the system store include an authenticated session token

Repeating this process results in the same results: no cookies presented to either mysite.example.com or auth.example.com, and no new cookies stored to either sites.

The GM_xhr response contains no set-cookie headers, either, but that's I think because I'm getting the response of the redirected page (the 401 error'd one)

Test 2: Stopping the redirect and checking for response headers

If I add "redirect": "manual" to my XHR parameters, I can see that the set-cookie header is present in the GM_xhr response, and the sso_rfp that was not set previously does get set. No cookies are presented to auth.example.com on redirect, though, so the 401 occurs again.

Test 3: Already-stored SSO Token Cookies

Now, I manual visit mysite.example.com. After the redirect loop succeeds, the system-stored cookies are:

Cookies for mysite.example.com include the SSO RFP and the SSO token
Cookies for auth.example.com unchanged

Example code

GM_xmlhttpRequest({
    method: "GET",
    url: `https://mysite.example.com/foo`,
    onload: myLoadHandler,
    onerror: myErrorHandler
});

Background requests

1. GET to mysite.example.com

Request headers

Accept: */*
Accept-Encoding: gzip, deflate, br, zstd
Accept-Language: en-US,en;q=0.9
Connection: keep-alive
Host: auth.example.com
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: none
Sec-GPC: 1
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36
cookie: sso_rfp=012346789abcdef; sso_token=<token>

The cookies stored in the manual call are presented and the request is authenticated without any redirection needed.

So really the issue I'm having is that the session cookie saved in the system store for auth.example.com is not being presented when the XHR request is redirected to auth.example.com. Perhaps that is the intent, and I am missing something...?

tophf commented 2 weeks ago

Maybe the extension should automatically retain the cookies for this userscript's subsequent GMxhr? Will this solve the problem?

derjanb commented 2 weeks ago

@grablair Thanks for reporting. Should be at 5.3.6200 (crx|xpi in review)

Please download the crx file linked above and drag and drop it to the extensions page chrome://extensions (after you've enabled 'Developer Mode'). For a quick fix please export your settings and scripts as zip or (JSON) file at the "Utilities" tab and import it back at the fixed BETA version.

@tophf Tampermonkey always stores received cookies in the browser's cookie store except when anonymous is set.

tophf commented 2 weeks ago

I see. So I guess the fact that a year ago Tampermonkey behaved differently (https://github.com/violentmonkey/violentmonkey/issues/1835) was a bug, while I incorrectly assumed it was the proper behavior. The userscript I mentioned in the linked issue logs out from the main account in the new beta TM 5.3.6200, just like it did in old VM before I changed it a year ago to match TM at that time. I guess VM will restore its old behavior too.

Note that the userscript didn't use anonymous: true in the old TM.

grablair commented 2 weeks ago

Thanks for the quick change, @derjanb !

I have installed v5.3.6200, but I am still experiencing the same issue it seems. When my XHR hits the target page (mysite.example.com) and gets the 307 redirect to the auth site (auth.example.com), no cookies are presented to the auth site despite being several being present in the system store for auth.example.com (including the session cookie which authenticates me).

derjanb commented 2 weeks ago

I see. Let me think about how to fix it...

For now you should be able to workaround the issue this way: ^{* works in BETA versions only}

let r = await GM.xmlHttpRequest({
    method:   'GET',
    url:      'https://auth.example.com',
    redirect: 'manual'
});

const h = {};
r.responseHeaders.split('\n').forEach(header => {
    const v = header.match(/^([^:]+): ?(.*)/);
    if (v) {
        const k = v[1].toLowerCase();
        h[k] = (v[2] || '');
    }
});

if (h.location) {
    // this request will include the cookie set by the previous request

    r = await GM.xmlHttpRequest({
        method:   'GET',
        url:      h.location
    });
}

taylorb-syd commented 2 weeks ago

@derjanb, could you please give us a summary of the intended behavior here? It would be helpful to know what the intended security model is you're using for this so that we can ensure any implementation is compatible with your extension going forward.

derjanb commented 2 weeks ago

@grablair Tampermonkey BETA 5.3.6201 (crx) should finally work.

@taylorb-syd The intended behavior is the one of Tampermonkey 5.1.1 (the last Manifest v2 version)

derjanb commented 2 weeks ago

So I guess the fact that a year ago Tampermonkey behaved differently

@tophf I can't reproduce this nor does the code show that the cookie header was filtered. It only added an own header to forward the value of set-cookie to GM_xhr. The only filtering that happened, was if the own header came from outside the extension.

rRobis commented 1 week ago

@grablair Tampermonkey BETA 5.3.6201 (crx) should finally work.

@taylorb-syd The intended behavior is the one of Tampermonkey 5.1.1 (the last Manifest v2 version)

I installed BETA 5.3.6201 and it solved cookie issue for me, but i'm wondering, if i will still get latest updates from store after updating it from the crx file?

Thanks!

derjanb commented 1 week ago

i'm wondering, if i will still get latest updates from store

@rRobis Yes, it signed by the Web Store and updates from there.

rRobis commented 1 week ago

i'm wondering, if i will still get latest updates from store

@rRobis Yes, it signed by the Web Store and updates from there.

But why the latest beta was not being downloaded when i clicked update button in chrome extensions, only if installed by crx file? (does it still maintain the store updates in that case?)

derjanb commented 1 week ago

Because it is still in review. After that the usual update works.

rRobis commented 1 week ago

Because it is still in review. After that the usual update works.

ok, thanks for clarifying

Tampermonkey / tampermonkey