search

Open-EO / openeo-api

The openEO API specification
http://api.openeo.org
Apache License 2.0
91 stars 11 forks source link

Authentication: The value of the 'Access-Control-Allow-Origin' header in the response must not be the wildcard '*' #41

Closed m-mohr closed 3 years ago

m-mohr commented 6 years ago

Seems we have a severe problem with Authentication: When sending Basic Auth credentials (probably also for Bearer token), the servers are not allowed to set the Access-Control-Allow-Origin to * (wildcard).

Not sure yet how this can be solved appropriately, maybe we need to have an alternative way for logging in, e.g. sending the credentials via body or query parameters?

Some background: http://blog.portswigger.net/2016/10/exploiting-cors-misconfigurations-for.html

m-mohr commented 6 years ago

Hopefully fixed with #42.

soxofaan commented 3 years ago

Can this ticket be re-opened (or should I create a new one)?

I think the solution introduced by #42 still leaves a security issue as discussed in that blog post https://portswigger.net/research/exploiting-cors-misconfigurations-for-bitcoins-and-bounties and https://ejj.io/misconfigured-cors/:

this combination as specified in the current openeo spec is problematic:

soxofaan commented 3 years ago

(FYI: while digging into Open-EO/openeo-python-driver#54 I stumbled onto this old ticket)

m-mohr commented 3 years ago

Unfortunately, I don't see a solution on the API side for the issue. Did you catch one? Everything that is more strict makes the JS client (including Web Editor / Mobile App / Jupyter Lab integrations) basically useless. The only thing I could imaging doing at the moment is to add that Access-Control-Allow-Origin should be properly validated (although I'm not clearly getting from the articles how to validate) and not return null or *. How do larger corporate APIs implement that in circumstances where the origin is not well-defined, i.e. is not a single web page?

Edit: Another unrelated thing we should probably add is that servers should send "Vary: Origin".

m-mohr commented 3 years ago

As far as I understand the Google API implementation (related SO comment), it doesn't allow requests from browsers (i.e. no CORS implemented) and you need to use the libraries hosted at google (i.e. same origin or white-listed origin). So that means if we want to make it work safely, every back-end needs to host their own JS client or it needs to white-list a CDN hosted JS client?! Wow!

aljacob commented 3 years ago

this is definitely an issue, and while setting this up in our environment a noticed, that there are many places where this creates issues and for sure leaves opportunities for security breaches, when working with the wildcard or not checking for CORS

soxofaan commented 3 years ago

The only thing I could imaging doing at the moment is to add that Access-Control-Allow-Origin should be properly validated

that is also the only thing I can think of at the moment. Possible implementations:

m-mohr commented 3 years ago

that there are many places where this creates issues

@aljacob Could you elaborate that a bit more?

whitelist of allowed origins

@soxofaan Is white-listing origins the only way out? I don't like any of the approaches to be honest. It makes browser-side usage a pain (includes many mobile apps) and how should the central instance (PSC?) decide who is trustworthy?

m-mohr commented 3 years ago

@jneijt How do you use the JS client at the moment? I guess this would break the mobile app as the Origin is always somehow the mobiles IP or so? Or is this centrally hosted somewhere at Solenix?

Some hosts that we'd need to whitelist:

soxofaan commented 3 years ago

is it necessary to whitelist these CDNs? as far as I understand you only have to whitelist the domain of a webapp "page" itself, not the domains of the JS code it loads.

m-mohr commented 3 years ago

That's how I understood the Google API workflow, but maybe it's indeed the site/apps origin, not the origin where the JS files are hosted. We'd need to invetigate... In case it's the site/apps origin, it gets even more annoying as we can't even offer a centrally hosted instance of the JS client for sites/apps to use. It makes the JS client very much unusable for externals and thus makes not much sense to put a lot of effort into it...

soxofaan commented 3 years ago

Is white-listing origins the only way out? I don't like any of the approaches to be honest ... how should the central instance (PSC?) decide who is trustworthy?

Indeed, a global, centrally managed whitelist is a pain, and it having the PSC maintain it doesn't scale well.

Ultimately it's all about the trust between a user (defined at backend side) and a webapp. So maybe it makes more sense to work with something like per-user whitelists, where a user maintains, through their backend account, a list of webapp origins it trusts. This makes webapp usage harder, but maybe we can think of a flow that doesn't raise the bar too much (e.g. bootstrap the whitelist with some defaults like editor.openeo.org).

Advantages:

Disadvantages:

Lot of disadvantages, so not a very likeable alternative either :worried:

m-mohr commented 3 years ago

I guess we would need both: User-defined origins and to get started a list of pre-defined origins that originate directly from within the openEO consortium. I don't see an alternative. (Unfortunately, I still don't completely understand from reading the articles what the exact security issue is. Can someone explain that?)

soxofaan commented 3 years ago

https://ejj.io/misconfigured-cors/ explains it somewhat better than the other blog post:

That's why it is important for backend.com to validate the origin in some way (e.g. whitelist or another back channel)

But now that I write down this flow, I realize that we might workaround the problem. The attack is possible because the browser automatically sends cookies along with the request from evil.com. But in case of openEO, we don't use cookies for authentication, but (somewhat custom) authentication headers that are set explicitly by the client code. So if we can specify that Access-Control-Allow-Credentials MUST be false (instead of MUST be true) , the problem might be solved without whitelisting. At the moment I'm not completely sure if Access-Control-Allow-Credentials is only about cookies and not about the authentication headers we currently send, that requires some more reading and testing I guess.

jneijt commented 3 years ago

@m-mohr I still have to read up on the articles you referenced but to quickly answer the question about what the mobile app does:

And on what @soxofaan said:

Ultimately it's all about the trust between a user (defined at backend side) and a webapp. So maybe it makes more sense to work with something like per-user whitelists, where a user maintains, through their backend account, a list of webapp origins it trusts. This makes webapp usage harder, but maybe we can think of a flow that doesn't raise the bar too much (e.g. bootstrap the whitelist with some defaults like editor.openeo.org).

Isn't that what Google does with it's APIs as well? When requesting an API key, you can/should set the allowed origins for the application you are going to use the API from. I think this is a pretty common approach.

m-mohr commented 3 years ago

https://ejj.io/misconfigured-cors/ explains it somewhat better than the other blog post [...]

  • evil.com has JS that does requests in the background to backend.com allowing it steal data (tokens, api keys, ...) because you are logged in (e.g. through cookies)

I've read that yesterday, too, but I'm still not sure it applies for us. How can someone evil making requests to backend.com grab information that he shouldn't get like the Authorization Header (he'd need to send, right?) or access tokens in responses? We don't have cookies that could be passed. Does it have something to do with the browser cache? And how does this "Vary: Origin" header affect that?

So if we can specify that Access-Control-Allow-Credentials MUST be false (instead of MUST be true)

AFAIK We can't do that because the header "Authorization", which we send the Basic and Bearer tokens with, can only be used in JS if Access-Control-Allow-Credentials is set to true.

Credentials are cookies, authorization headers or TLS client certificates.

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Credentials But as I said, not sure how the Authorization header is then exposed to evil.com...

m-mohr commented 3 years ago
  • As far as I know, it uses localhost:4200 as origin when requesting anything on the backend (development for sure, I thought this is true for the production one as well).

Would localhost make problems with the stricter approach? I'm not sure...

Isn't that what Google does with it's APIs as well? When requesting an API key, you can/should set the allowed origins for the application you are going to use the API from. I think this is a pretty common approach.

I know you need to specify redirect URIs for OpenID Connect, but I'm not aware that you can specify CORS Origins... anyone has details? Can only find information for cloud buckets, not for APIs.

is it necessary to whitelist these CDNs?

Doesn't seem to be the case. I just tested it and it is NOT using the host of the JS file, but the host of the requesting web page as origin. Then I don't understand yet how Googles APIs work... Edit: Actually, https://de.tenable.com/blog/understanding-cross-origin-resource-sharing-vulnerabilities says you should not allow CDNs.

jneijt commented 3 years ago

I know you need to specify redirect URIs for OpenID Connect, but I'm not aware that you can specify CORS Origins... anyone has details? Can only find information for cloud buckets, not for APIs.

Isn't this done by specifying "HTTP restrictions" on the API key? It's called differently, but I think it's basically the same idea. I might be completely wrong here though... https://cloud.google.com/docs/authentication/api-keys#adding_http_restrictions

Would localhost make problems with the stricter approach? I'm not sure...

Which stricter approach do you mean? AFAIK it has to be added to the allowed origins, just like any other URL like editor.openeo.org.

m-mohr commented 3 years ago

As far as I understand https://www.sjoerdlangkemper.nl/2018/09/12/authorization-header-and-cors/, Authorization headers are attached automatically to requests in some cases (i.e. Basic Auth), so that users don't need to input the credentials each time the click on a new page. This would indeed expose credentials to evil.com, but in our case I hope this is not the case and Bearer Tokens are not sent automatically. Still, it would break the security of /credentials/basic, but the API clearly says it is not meant to be used for production so I'd be fine with that.

Edit: SO question on which Authorization schemes (e.g. Basic, Bearer, ...) are sent automatically by browsers, unfortunately no response yet: https://stackoverflow.com/questions/15427650/when-is-the-authorization-header-automatically-sent-by-the-browser

Isn't this done by specifying "HTTP restrictions" on the API key?

In the English version it explicitly names it "HTTP referrers". Does this refer to HTTP Referrers (i.e. header Referer) or the HTTP Header Origin or both? If it's just HTTP Referrers, it doesn't apply for our use case here.

Which stricter approach do you mean?

White-listing specific domains instead of just reflecting Origins.

Current status for changes in 1.0.1:

jneijt commented 3 years ago

White-listing specific domains instead of just reflecting Origins.

As long as localhost is whitelisted, it should be fine. But I don't know what implications whitelisting localhost has.

m-mohr commented 3 years ago

As long as localhost is whitelisted, it should be fine. But I don't know what implications whitelisting localhost has.

Yeah, if * is an issue, localhost should be black-listed as everyone could send requests from localhost, right? Maybe I'm missing something though...

jneijt commented 3 years ago

That's my feeling as well, exactly...

jneijt commented 3 years ago

Ok, it looks like the Origin is a matter of configuration for the mobile app. By default it uses ionic://localhost on iOS and http://localhost on Android but both the scheme and the hostname can be configured according to the documentation (https://github.com/ionic-team/cordova-plugin-ionic-webview#hostname).

I haven't tested this but it would solve the localhost issue for the mobile app as we could define some origin to use for the app.

m-mohr commented 3 years ago

~@jneijt Could you specify a distinct domain name in the app for the next release so that if we decide to implement white-listing later, we can easily add the domain? Either some solenix domain name or something like https://demo-app.openeo.org ...~ Might not be required... see below.

Regarding the security of HTTP Basic, it seems some valuable information can be found here: https://tools.ietf.org/html/rfc7617#section-2.2 I couldn't find anything similar in the Bearer RFC yet and I doubt it's in there as it actually is written for OAuth: https://tools.ietf.org/html/rfc6750

SO question on which Authorization schemes (e.g. Basic, Bearer, ...) are sent automatically by browsers, unfortunately no response yet: https://stackoverflow.com/questions/15427650/when-is-the-authorization-header-automatically-sent-by-the-browser It seems though his experiments lead to the conclusion Bearer Token are not sent automatically by browsers. So maybe we are safe?

soxofaan commented 3 years ago

How can someone evil making requests to backend.com grab information that he shouldn't get like the Authorization Header (he'd need to send, right?) or access tokens in responses? We don't have cookies that could be passed.

Indeed, that is also my current understanding of the attack: it depends on the browser to automatically include backend.com cookies with the requests done from evil.com. As far as I know there are no browser mechanisms that automatically include authorization headers to JS-based requests in the same way. So the security issue might not apply to openeo API (as long as we don't use cookies).

As far as I understand https://www.sjoerdlangkemper.nl/2018/09/12/authorization-header-and-cors/, Authorization headers are attached automatically to requests in some cases

Ok interesting, so there are mechanisms to automatically include authorization headers. Not sure if this applies to requests done from JS as well.

So if we can specify that Access-Control-Allow-Credentials MUST be false (instead of MUST be true)

AFAIK We can't do that because the header "Authorization", which we send the Basic and Bearer tokens with, can only be used in JS if Access-Control-Allow-Credentials is set to true.

I'm not sure here. The client indeed sends the tokens through headers to the backend, but it does not receive them from the backend:

so we don't need to enable Access-Control-Allow-Credentials=true I think.

I've been playing a bit with that demo page of https://www.sjoerdlangkemper.nl/2018/09/12/authorization-header-and-cors/, and it seems that it indeed works without Access-Control-Allow-Credentials

m-mohr commented 3 years ago

@soxofaan Interesting, from reading the page it seems that the header is required if you want to send Authorization headers (that's how I understand it), but the demo page indeed doesn't require me to add the Credentials flag... strange, I'm pretty sure I struggled with this in the JS client. I'll so some experiments tomorrow with GEE and JS.

m-mohr commented 3 years ago

Okay, looking into the code of the demo page and reading some additional docs enlightened me! (I hope I understood it correct.)

I need to disable withCredentials: true (the same as credentials: include in fetch) in the JS client, but still send the Authorization header! I thought withCredentials would be required to send the Authorization header, but it doesn't seem to be the case. I'll check that tomorrow, but in that case we can completely omit the Access-Control-Allow-Credentials header and just specify * for Access-Control-Allow-Origin. That makes it sooooo much easier!

@jneijt Forget my request above for now...

All this might be going into a 1.0.1 release with PSC vote before.

m-mohr commented 3 years ago

It works like a charm without the credentials headers etc. So I'll make PRs, push it through PSC if required etc. So that I hope to get an API 1.0.1 out soon. Thanks for pushing me on this, @soxofaan! This clearly shows how bad documentation can influence a whole project.

@jneijt I'll also issue a fix for the JS client, so better wait a day with the release so that you can get in those changes.

jneijt commented 3 years ago

@m-mohr Great to hear, thanks! I'll wait until the new version is available ;-)

m-mohr commented 3 years ago

Two PRs open: https://github.com/Open-EO/openeo-js-client/pull/38 and https://github.com/Open-EO/openeo-api/pull/345