When ModSecurity/naxsi and auth_request (Vouch Proxy) and HTTP/2 is enabled, POST/PUT requests hang

[jangrewe]

Describe the problem First of, thanks for creating vouch-proxy, it works - and it works great! I'm using it with Nginx and Authentik as the IdP, and everything works as expect... well, nearly everything.

There are some tools that i protect via SSO using vouch-proxy, e.g. RabbitMQ Management Interface, ElasticHQ or Stable Diffusion, and they work fine without SSO enabled. Once i do enable SSO via vouch-proxy, some functionality suddenly breaks.

When having a look at the Firefox DevTools, i can see all requests getting fired off, but they don't do anything. In the request details, there are no info for Headers, Request, Response or anything.

They don't even seem to make it to the server, as least i can't see anything in my access logs, and not even in vouch's logs (with vouch.loglevel: debug, of course). I can see all other requests, but not those particular POST/PUT request. So i also can't provide any logs, because there is nothing to be shown. 🙁 When enabled vouch.testing: true, there is also nothing to see - because the request never "leaves the browser".

So i assume this is a browser-side issue, but i have no idea what's wrong there. It's gotta be related to the cookie, and that this blocks Firefox from even sending the request properly. Oh, this happens in both Firefox and also Chrome, so it's not vendor-specific.

My config matches config/config.yml_example_oidc with secure:true set, so nothing fancy.

Expected behavior My browser sends all request as without SSO enabled.

Desktop (please complete the following information):

• OS: Win11
• Browser: Firefox 120.0.1, Chrome 120.0.6099.71

Additional context All potentially blocking extensions disabled in Firefox, no extensions installed in Chrome.

[bnfinet]

@jangrewe thanks for the kind words about VP. Could you please read the README and provide details and configs and logs as described there, at least your VP config and NGINX and perhaps the nginx logs that lack the traffic.

[jangrewe]

Sure, here you go:

vouch config: https://hasteb.in/zWQ1Qb7nFzPmgmI nginx sso config: https://hasteb.in/sYaMKjlafqXohY5 (this gets included into the vHosts server{}, outside of location / {} nginx vhost config: https://hasteb.in/4VQldj5mGnMp3Ku vouch log: https://hasteb.in/BijCuNdQ95bpoUB

And a screenshot from the DevTools, where you can see the "stuck" request:

So the only difference between "werky!" and "no werky!" is wether sso.conf is included. If it's not included, everything's peachy. But wide open...

[bnfinet]

try setting cookie.httponly: false and possibly cookie.secure: false

https://developer.mozilla.org/en-US/docs/Web/HTTP/Cookies#security https://github.com/vouch/vouch-proxy/blob/master/config/config.yml_example#L128

I'm wondering about your local networking pointing at the docker instance running the Elastic container. If it's going directly to :9200 that seems to be evading :443 which is where VP is operating. Is that accurate?

if that doesn't work please include the rest of your nginx configs and I'll take a look

    include snippets/global.conf;
    include snippets/ssl.conf;
    include snippets/modsecurity.conf;

[jangrewe]

Setting vouch.cookie.httponly: false and vouch.cookie.secure: false didn't make a difference, unfortunately.

You can ignore the :9200, that's just the frontend telling the backend where to connect to, as that's not connectable publicly (which would be a bad idea). This also isn't Elasticsearch, only ElasticHQ, a third-party interface... but yes, the ElasticHQ container would connect to the ES directly from one container to the other, without going through Nginx.

So please ignore any of that, the issue is the request that would tell the backend where to connect to, if it was ever actually sent! ;-)

global.conf: https://hasteb.in/zvp0brBhlzdQHck ssl.conf: https://hasteb.in/k0mqiNI9yPcRn3q modsecurity.conf: https://hasteb.in/Bt8o6LeAH0PgPyv

But fret not, ModSecurity runs with SecRuleEngine DetectionOnly, so it's not interfering with anything (yet!).

[bnfinet]

Where's the vhost config for sso.faked.org?

Feels like Access-Control-Allow-Origin stuff. Is that getting set by Elastic or nginx somewhere?

Though I'd expect the browser to complain about origin stuff.

Is it just hanging? That feels like DNS and networking.

Could also be the XHR call with allow-credentials not set.

https://stackoverflow.com/questions/21850454/how-to-make-xmlhttprequest-cross-domain-withcredentials-http-authorization-cor

If it never arrives at Nginx it's hard to solve it with Nginx and VP.

[jangrewe]

The vHost for sso.faked.org is also pretty simple: https://hasteb.in/MlwnJ9QwyR1LxFX

There are no warnings in the browser console about anything - and please ignore Elastic, as that's just one of many examples. I've suspected "something else" at first, too... but then it would also happen if i comment out the sso.conf that does the auth_request call. Even just commenting out the auth_request /validate in sso.conf makes everything work perfectly fine - though without any protection then, of course. So i'm rather confident that it's not DNS or networking. It's all on the local network anyways.

So it's definitely something that causes to not send these particular requests, typically POST or PUT, but also not all of them.

Here's another request example with headers:

[bnfinet]

Is it just hanging? The browser tells you nothing?

Can you right click on the request and copy it as curl? Does it look well formed? What happens when you execute that from the command line?

[jangrewe]

Sorry for the delay in responding...

So i copied the request for curl, and yes, the request then just hangs: https://hasteb.in/uJaxxOCYRPypTke

Only AFTER i CTRL+C the request, i get some log output from vouch: https://hasteb.in/EhKEMM4Kky7OstZ

[jangrewe]

Okay, i think i solved it. When searching around a bit more (and more generic), i came accross this: https://trac.nginx.org/nginx/ticket/801 The conclusion being:

That is, it's a naxsi bug.

A quick search shows me that "naxsi" is a WAF module similar to mod_security, and the explanation given in the ticket sounded pretty logical.

... so i disabled mod_security completely, et voilà, everything works again. So i think it may be worth pointing out that there are seemingly harmless modules that will break auth_requests if they hook into the request.

Thanks for sticking with me on this one, i'll see if i can find a way to keep mod_security enabled without breaking auth_requests. 😄

Edit: https://github.com/SpiderLabs/ModSecurity-nginx/issues/163 maybe also https://github.com/SpiderLabs/ModSecurity-nginx/issues/130

[bnfinet]

Thanks for the details on the root of the problem.

Could you clarify, you're saying that you have a bespoke mod_security setup? I think your are saying that your Nginx is compiled to add this module...

https://github.com/wargio/naxsi

Is that correct?

[jangrewe]

Yes and no. I'm using ModSecurity/mod_security: https://github.com/SpiderLabs/ModSecurity-nginx I am not using "naxsi" and haven't actually looked too deep into it, i just saw that it's "something like ModeSecurity".

As that ticket mentioning "naxsi" talks about accessing requests and not returning them as expected, i figured that ModSecurity may after all be an issue, even though it's running in DetectionOnly mode.

So it's not the functionality of what ModSecurity does, but how it does it - and that's always the same, even if DetectionOnly is set. Or any other 3rd-party module that does similar things.

[bnfinet]

Thanks for the clarification!