Follow 302 redirects when fetching /alerts.json

[BenoitKnecht]

I use Karma with alertmanager.proxy: true and authentication/authorization handled by a reverse proxy (Cloudflare Access to be precise).

The issue is that the session tokens set by Cloudflare Access expire after a while, so when Karma tries to fetch from /alerts.json, it gets an HTTP 302 reply and stops there. If it followed the redirects instead, it would be able to renew its token (through the magic of SSO), and then it could fetch the alerts as if nothing had happened.

Does that make sense?

[prymitive]

That should be easy to add, redirect: 'follow' is needed on fetch() calls as per https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch

The only potential problem is that I don't know how credentials are handled with a redirect, so not 100% sure if this will fix the problem, but adding this is essentially a one-liner, so let's give it a go.

Will look into that, thanks!

[prymitive]

Raised #1158 that enforced redirects, but looks like that's already default for at least Google Chrome 47+ so I'm not 100% sure if that's enough

[BenoitKnecht]

Hmm, interesting. I'm using Firefox, but I think a colleague had the same issue on Chrome 69; so maybe you're right and that's not enough.

In any case, thanks for the very quick reply, and more generally, thanks for this awesome tool!

I'll try and find the time to test your patch, see if it really fixes this issue.

[prymitive]

Latest docker image should have this change if you wait 30-60 minutes. I suspect that fetch requests are not triggering Access revalidation, but need to look a little more into it

[prymitive]

If I go via Access and remove the auth cookie the next fetch errors with:

Access to fetch at 'XXX' from origin 'XXX' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

So I think the issue is that when Access gets into the middle of request it doesn't have same CORS headers (since it's not part of karma service, so lacks same setup). This can be fixed by passing mode: same-origin or mode: no-cors to fetch().

It feels that same-origin would be best here, given that karma UI & API are intended to be run as a single service on the same domain. I'll look a bit more into that, changing that is another trivial PR so it just needs a bit of testing.

[BenoitKnecht]

You're too fast, I haven't even had the time to do a Docker pull :)

Good to know that there's a simple fix, I'm looking forward to it!

[prymitive]

Well, there's a redirect involved going through Access, so it won't be same origin. Need to test if no-cors really fixes it and whenever it doesn't break anything else. This might be more tricky than I expected

[prymitive]

I think the solution is to use no-cors for /alerts.json, as it's a simple GET and that will allow it to follow the redirect, but for anything that does A POST we need to use "full fetch" with CORS mode.

[prymitive]

Another option is to set redirect: "manual" and if the response is type: "opaqueredirect" then we could reload the page to let use re-authenticate

[prymitive]

1159 splits fetch call so GET requests can use different options from POST or DELETE, hopefully this will play better with Access

[prymitive]

Please try :latest docker image in 30-60 minutes

[prymitive]

no-cors — JavaScript may not access any properties of the resulting Response

Missed that, with no-cors we won't get the body, which kinda makes the whole effort of making a request pointless. That leaves us with redirect: manual and reloading the page whenever we start getting 302 instead of 200 from fetch

[khauser]

We run into this issue as well. Instead of "Cloudflare Access" we have keycloak and an oauth2-proxy next to karma. At the moment I'm manually reloading the page whenever the fetch fails.

[prymitive]

I don't belive that there's anything we can do to refresh auth sessions while doing fetch (by following redirects or something else), especially that some auth implementations might ask for use input (input 2nd factor auth, choose the account to use, etc) so the best solution I can think of right now is:

• have retries for failed fetch requests (don't wait the usual 30s or so but retry faster)
• after a few consecutive failures reload the page

Will try to add that soon

[prymitive]

I've added fetch retries, which will cover temporary issues, so now if we bubble a fetch error it's only if we tried 6 times (first + 5 retires). This should make it easier to add failure handling. I'll try to experiment with popup windows - if we detect that fetches are failing we could open a browser popup window that will make a request and close, that should refresh the session, and if it asks for any user input it will happen without affecting main window. Not sure if that will work well since I browser might be blocking pop up windows that are open without user click.

If that doesn't work reliably implementing a full page reload is easy.

[prymitive]

Popups seem to be blocked unless they're raised after user click, so don't think I can use that.

Raised #1285 that should help with this problem - it will add more retries to fetch and if it keeps failing it will eventually switch from mode: cors to mode: no-cors, which (at least with cf access) should prevent requests from being blocked at the early stage and allow to inspect the response. The idea is that if cors requests are failing, but a non-cors goes through then there's something in between the browser and the backend, so we should reload the page. This might need a few tweaks but seems like might work without too many false positives.

[prymitive]

Please try v0.53 release that I've tagged recently. I did test it with Cloudflare Acces and it does correctly reload the page when I delete session cookies while karma is open. Not sure if that will work for all auth proxies and/or brower so please report if it doesn't work. Thanks!

[khauser]

For keycloak it seems to work also. No manual reload anymore needed... Thank you very much. You earn your Christmas gifts very well!

[b4ldr]

Sorry to drag up an old issue however im seeing a similar problem using Apereo CAS as the SSO identity provider and mod_auth_cas to provide authentication to the karma UI. The error i see in the chrome console is

Access to fetch at 'https://alerts.wikimedia.org/alerts.json?&gridLabel=severity&gridSortReverse=0&sortOrder=&sortLabel=&sortReverse=&q=%40state%3Dactive&ticket=###redacted###'
(redirected from 'https://alerts.wikimedia.org/alerts.json?&gridLabel=severity&gridSortReverse=0&sortOrder=&sortLabel=&sortReverse=&q=%40state%3Dactive') from origin 'https://alerts.wikimedia.org' 
has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. 
If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

From my own testing im still able to use the UI correctly and wonder if the error in the google chrome console is an expected artefact or if we need to configure something else?

more details can be seen in our bug tracker

[prymitive]

Yes, errors in console are expected. Here's what happens:

• You open karma.example.com with some proxy service between your browser and karma itself
• Auth proxy in between will redirect your browser on the / request to authenticate and issue some form of an token
• If you leave browser running eventually that auth token expires
• Next request for /alerts.json will be intercepted by proxy auth, instead of passing it to karma backend auth proxy will try to redirect it some auth page - this often breaks CORS since that auth page will likely be under different origin than karma.example.com (like sso.example.com) or is missing CORS headers, which means that the browser will block such request, /alerts.json requests will fail, plus it's a XHR request so it won't render that auth page in the browser anyway
• The result is that karma will keep failing to fetch data from /alerts.json until you reload the page, which triggers in-browser redirect to the auth page again, the loop restarts

This is really poor UX you'll get from auth proxies, any web service routed via one will behave in a similar way.

To minimise the disruption karma tries to detect this situation and help those auth requests complete by disabling CORS headers after N number of failures. Disabling CORS headers means that browser won't block the request, it will get redirected to the auth proxy and there's a good chance it will trigger token refresh, which will then allow subsequent requests to work. But since we first need to observe some failures to know that things ain't right and it might be time to try without CORS you will see some failing requests in the console.

[b4ldr]

Thank you for the super quick and detailed response