Closed f0zi closed 9 years ago
I recommend reading https://wiki.whatwg.org/wiki/FAQ#Is_there_a_process_for_adding_new_features_to_a_specification.3F Feel free to use this issue instead of the WHATWG mailing list though.
Thanks, I missed that. So let me add a use case:
A webapp is using distributed services and wants to protect the user against MITM attacks.
On startup, it gets a configuration document from a discovery server using per call certificate pinning:
fetch({ url: "https://example.com/.well-known/appconfig", certs: [ "<hardcoded discovery fingerprint>" ])
From this document it gets the fingerprints for the servers it communicates with and registers them globally:
fetch.addFingerprints(config.fingerprints)
This is a rather generic use case that could be added to any app that is using fetch e.g. for authentication, CDN, 3rd party services, etc. to add certificate pinning without changing the way it does web requests.
As the certificates that have been used for HTTPS are not exposed to the client there is no way of achieving this with the current interface.
Something I missed yesterday is a case where you have a global list of fingerprints but you do a request to a host that is not on the list. This might be ok, or ok it the server uses HPKP. This would require some global default flag that would be "strict" by default but could be set to something like "allow-unknown" or "allow-unknown-hpkp".
Another use case would be services like OpenID, which could include cert fingerprints in their config document which could then be used by an OpenID client to do cert pinning.
Those are not use cases. What problem are you trying to solve? (Avoid pinning / fingerprints in the answer.)
A web app is using distributed services and wants to protect the user against MITM attacks. Considering that fraudulent certificates have been issued before it's not enough to rely on the browser's or the OS's root certificates to validate the certificate chain.
This is the actual requirement that we have for our app (which does federated authentication). My first use case is what we have implemented at this point.
I don't think this is going to work from client-land.. With a JS API any first- or third-party script would be able to inject arbitrary pin rules for any origin - this is bad, as it's trivial to abuse. Only the origin itself should be able to assert rules about which certs should be pinned and for how long, which is why header based registration works.
@igrigorik I agree that the global list should be per page instance and per origin, kind of like the current page's cookies are.
But if you say that it's useless because theoretically you can compromise it than one could say the same about HPKP or WBC.
If someone does MITM then replacing the HPKP headers is trivial and would fool any browser that did not see this domain before. Accessing a 3rd party service is doing exactly that, already knowing the fingerprint would allow the detection of the MITM even on the first access. I'm not trying to replace HPKP, but complement it.
As for the attack target, it is more efficient to do a MITM against the 3rd party service and get all connections to it then to MITM all it's clients and have to change the JS for each of those individually. The amount of compromised users is smaller and the effort is higher - the bar is raised. I believe that's a good thing.
Also consider the case where the app is actually deployed at the client in a secure way, e.g. in a signed installer so it does not have that problem. The fingerprint it would use for the discovery server is in that package.
@f0zi no, there is a big difference here. With HPKP only the server responsible for particular origin is able to assert what pins are valid; a different origin cannot and should not be able to enforce arbitrary rules on other origins.
Also consider the case where the app is actually deployed at the client in a secure way, e.g. in a signed installer so it does not have that problem. The fingerprint it would use for the discovery server is in that package.
Packaged apps are a separate beast, they're subject to particular deployment model, etc. Out of scope.
@igrigorik is there a pointer for why a different origin should not have that ability?
@igrigorik Look at Google's OpenID config document, it is cross domain and could very well provide a set (or two, as recommended by HPKP) of valid fingerprints for the endpoints.
@annevk seems like it would be trivially abused to block access to origins (e.g. some third party decides to block access to competitor.com by specifying false list of pins).. and in the process it would trigger false cert validation failures which is a terrible outcome both for users and owners of that origin; if origin changes their cert they block themselves because other origins are using old pins in their configs.
That said... maybe I'm barking up the wrong tree. I'll defer to @sleevi and @mikewest.
@igrigorik I would hope that what @f0zi refers to as "global" is actually scoped to the origin of the application.
Ah, if its scoped to own origin only then that's more reasonable.. I read the proposal as "my app wants to specify pins for origins I depend on". That said, even for same origin, what happens when a third-party script runs on my site? Would it be able to clear and modify pins for my origin.. because that doesn't seem right either.
The way I read the proposal is that you can pin the certificate for certain origins your origin might connect to. Basically putting additional requirements on cross-origin requests.
Third-party script seems like a red herring. No different from a first-party script. An origin must not have the ability to change its own pins through script.
with my myopic, close-minded, "no is the best answer," hat on, I'm inclined to say "No, we shouldn't support this."
As @igrigorik lets on, this is rife for abuse. Some of the general limitations of this have already been discussed in past discussions related to S-Links, but having client-specified policy for arbitrary origins is generally quite problematic (it's the SRI problem, except at a whole new level). The opportunity for misuse / improper configuration alone should doom this proposal.
But anyways, back to the abuse case. The end result of such a proposal is that pools of sockets would no longer be able to be pooled by destination origin - that is, a pool of say six sockets to (https, google.com, 443) - because it would have to consider the calling origins effective-client-policy at the time of connection. That is, it'd minimally be a tuple of (destination origin, source origin, security policy), since the entire point of such policy is that it affects connection establishment.
Now, you could say something like "Oh, no, just look at the existing sockets, and if one doesn't meet the specified policy, close one of the existing sockets and establish a new one" - except that presumes that user agents implement a 1:1 mapping of socket<->request (they don't; socket connections are vended out independently of the requests that require them), and it's really just an expression of the concept of a "shared socket pool", where you're still isolating the sockets on the axis I mentioned, it's just broken into subpools. That is, (destination orgin) is key 1, (source origin, security policy) is key 2. The disconnect&restablish just keeps the connection limits keyed off key 1, but still has to account for key 2 in the actual establishing.
It also upends a bit of the priorities of constituencies here, although in a way that I feel almost glib for using that phrase. Here, the destination-operator should be able to set their security policy as they see fit, for they're the operator. They might choose CA 1, but then later, decide for economic reasons that CA 2 is far more acceptable. By allowing source-operators to control how that security policy is applied, the source-operators can (effectively) try to interfere and prevent the destination-operator from doing that. This is by design - the source-operator wants to restrict the ability of an evil MITM to swap the certs out, but as a consequence, the source-operator end sup dictating policy for the destination-operator, forcing the destination to either accept "This is the CA we must use" or simply break those links. There's no question in my mind that source-operators are far, far less likely to update their links than destination-operators, and so this just encourages bad linking. This is the s-link problem.
Such pins are inherently non-static. CA lists change over time, someone will have to be curating all those outbound pin sets to reflect modern practice. That's an unusuable nightmare for outbound links; and is just sustainable enough when maintaining HPKP/HSTS.
Like I said, I understand where this is coming from, but I don't think it's good at all. I've argued against such proposals in the past (e.g. S-Links) because I think they offer even larger footguns than HPKP/HSTS.
And I'm particularly not taking the time to explain why the individual directives proposed are themselves problematic, but I do want to highlight that even if these concerns are ignored in favour of exposing this (after all, I am someone who says "no" a lot), there's still real issues with the nature of what's being requested.
Oh, and since I didn't address the counter-proposal (surface the details to the app), I'll just note
private
flag / strict
mode suggested). A user in charge of their machine necessarily trumps the needs of the website, whereas cert pinning that allows a site to block such self-MITM violates that priority. No matter the views of the site (which may not necessarily be the site being accessed, to be cleared), the user should be allowed to freely operate their machine, which may include MITM'ing themselves for debugging (e.g. Fiddler) or for 'security' (e.g. antivirus).@igrigorik Yes, with global I meant per origin. As @annevk mentioned, it would be a more strict requirement on cross-origin requests.
@sleevi Thanks for taking your time, you have made a lot of good points.
The one I think is a non-issue is the connection pooling part. Connections in a pool are connected to a specific host and have the TLS handshake done. That means there is a cert attached to it that either passes or fails. You are also very correct that after the request has been sent it is too late for the check as the connection would have already leaked information. (You actually pointed out a bug in my implementation) So if a request takes a connection out of the pool it would check it's own (origin based) list against the cert fingerprint. If it fails to match the request should be failed, establishing a new one is not an option (and the connection could go back to the pool, although at this point I would say that at least from one origin's point of view we are connected to an evil MITM).
So the source origin and the security policy don't affect the connection pooling. Also the fact that the cert fingerprint has to be checked early (before sending the request) means that there is little hope on monkey-patching this in later, it has to be done by the implementer or it can not be done securely. Note that I believe that this is an opt-in feature that will not be used by many applications.
I have to agree with the points you made about the origin forcing a cert on the destination. This is for sure not something that a site operator should apply to all 3rd party services, otherwise links will be broken. Again, I don't think this is a feature that should be applied on all web apps and I don't really have a good answer, however let me present you with two scenarios that would mitigate this problem:
1) The 3rd party is not a 3rd party but simply just another domain of the 1st party or there is a close relationship (like a contractual agreement, think paid service) to the 3rd party. In this case one can assume that the fingerprints (as recommended by the HPKP, a set of two, the current one and a backup) should be up to date.
2) The 3rd party publishes (e.g. though HPKP) two sets of keys, and the 1st party updates those regularly, automatically, according to the caching rules. While this might fail if the MITM completely cuts off the 3rd party from the internet it will still work for the case where the MITM can only affect a section of the internet (e.g. in a country) and both the 1st party and the 3rd party are not in this section while the client is. Services that could make use of this could be authentication services or public communication services.
As for your points on the counter proposal, I have to agree with you, at the point the request went through it is too late and the cert could mean information leakage. I agree that considering this it's not an option.
I also agree that the user should be in control of their machine, unfortunately more often than not it is not only the user who is in control of the root certificates. This is part of the problem we're trying to solve: How to detect and prevent usage of a fraudulent certificate that seems completely valid from a chain-of-trust point of view. While I want the user to be in control of the data he is sending out I would also argue that e.g. an OpenID provider should be allowed to deny the user the successful authentication if the user is unable or unwilling to securely connect to the authentication endpoint.
Given that a request should not be sent at all if it does not match the expected certs I now think that the whole flagging idea is pointless. So the four modes should be:
All but "Off" would fail the request if the server cert is not matched against a fingerprint.
Also, to provide a lighter version and less of a footgun the global list could be left out. The mode flag would be unnecessary, all the request would need is a list of fingerprints that would have to be provided per request. If the list is provided the cert would have to match, otherwise it would behave as before.
At this level of support I think a global list could be monkey-patched in, if required.
The one I think is a non-issue is the connection pooling part. Connections in a pool are connected to a specific host and have the TLS handshake done. That means there is a cert attached to it that either passes or fails.
No, this isn't correct. Connections might be pooled across multiple hosts (e.g. due to HTTP/2 Connection Pooling or HTTP/2 Alt-Svc, Session cache pools, etc).
For example, imagine ssl.example.com is serving Cert A via SNI, and it has Hash A. Now imagine foo.example.com is serving Cert B, also via SNI, which has Hash B. foo.example.com has a SAN that covers ssl.example.com, and so it is viable to be connection-pooled. That is, if a new request for ssl.example.com comes in, we can reuse the connection to foo.example.com. If we then reject it (e.g. because the App inappropriately pinned Hash A), then if we reconnected, we could get Cert A, because we wouldn't be re-using the pooled connection. This is but one example of many, and it's why I said that it would require the pools to be sharded.
So if a request takes a connection out of the pool it would check it's own (origin based) list against the cert fingerprint. If it fails to match the request should be failed, establishing a new one is not an option (and the connection could go back to the pool, although at this point I would say that at least from one origin's point of view we are connected to an evil MITM).
And I think this is explicitly bad behaviour, which is why I have no desire to support it. There's a lot more edge case here that I think is well in the realm of "unspecified but common" behaviour, but I think the fundamental idea is unfortunately impractical.
Note that I believe that this is an opt-in feature that will not be used by many applications.
I think that alone is reason to be somewhat suspicious. It's opt-in, limited use, high footgun, violates layering, adds complexity, and for what value? What alternatives exist? HPKP, certainly :) CT, arguably.
unfortunately more often than not it is not only the user who is in control of the root certificates. This is part of the problem we're trying to solve:
Right, here's where our fundamental disagreement will be readily apparent. I disagree with the philosophy that the browser can prevent code running on the same native OS, with the same or greater capabilities, from interfering. To that end, we even wrote up the Chromium policies we use to evaluate such requests to provide greater clarification of the general opposition - https://www.chromium.org/Home/chromium-security/security-faq#TOC-Why-aren-t-physically-local-attacks-in-Chrome-s-threat-model- and https://www.chromium.org/Home/chromium-security/security-faq#TOC-How-does-key-pinning-interact-with-local-proxies-and-filters- . The OpenID argument is a very similar argument to that made re: autocomplete="off" and letting sites subvert the user's wishes, and that's covered in https://www.chromium.org/Home/chromium-security/security-faq#TOC-Why-does-the-password-manager-ignore-autocomplete-off-for-password-fields- (and in general, is a violation of the priority of constituencies)
I totally understand where this feature is coming from - this is something like I said has been explored before (S-Links is one example, but you will find the argument goes back nearly a decade). I think the idea represents a serious footgun and violates the separation of principals - there's no way the user agent can know whether the site meets your criteria or not (even DNS is not sufficient), and the effects on effective connection management alone should be sufficient for an argument against it.
@sleevi I still think your connection pooling argument is void, even if you use SNI you can't pool connections together with other connections to different domains even if they connect to the same host, cert pinning or not. If you do you are connecting to domain B with domain A's certificate (because that's what you got from the pool) without ever seeing domain B's cert. Reconnection or at least renegotiation would be necessary anyways, and at that point cert pinning can happen.
I'm not sure what exactly you are referring to with "explicitly bad behaviour". I was being sarcastic with the evil MITM.
I agree that this feature is limited use but I also believe that it is also an essential security feature for a small set of use cases that can't be done otherwise. I'd like to work together with you and everyone else to work out an interface that is less of a "footgun".
As I wrote in my first post, this is similar to HPKP but it is using a different angle so I think it's complementing it.
I don't think your comparison with the user passwords is valid. The user's password is the user's data and he should be in charge of it. However, if I provide a service to the user (like OpenID) then the user is not entitled to the service at any cost, I can set up rules and regulations for the usage of it. If I decide that I require the user to directly connect to my authentication endpoints than that is my right as service provider and so is my right to deny the user access to my service if he does not comply on the grounds of being unable to provide the security that the users would expect. I think this is actually good for the user.
You also seem to have misunderstood what I said about the root certificates. What I meant was not malware or tools like Fiddler running on the user's machine, I was referring to root certs that have been compromised, like the Comodo or the Diginotar incidents or simply certs I'd rather not trust.
even if you use SNI you can't pool connections together with other connections to different domains even if they connect to the same host, cert pinning or not.
This is based on incorrect understanding. @f0zi , you can read https://tools.ietf.org/html/rfc7540#section-9.1.1 for some of it, but everything I previously said still stands. It's absolutely valid to pool connections to different domains, so long as they have certs covering the alternative domains. Browsers have been doing this for nearly a decade.
re: explicitly bad behaviour, I mean the idea that the application is making this decision on behalf of an arbitrary third-party origin it does not control or operate. It's an inversion of control, and one with profound implications. I'm saying your use case is not valid.
I agree that this feature is limited use but I also believe that it is also an essential security feature for a small set of use cases that can't be done otherwise.
You've not established a use case where HPKP doesn't address, other than those that intentionally violate the priority of constituencies (anti-user) or invert control of security policy.
If I decide that I require the user to directly connect to my authentication endpoints than that is my right as service provider and so is my right to deny the user access to my service if he does not comply on the grounds of being unable to provide the security that the users would expect. I think this is actually good for the user.
I'm sorry, but this statement is unfortunately an example of an explicit and intentional violation of the priority of constituencies. The user's ability to access the service as they wish - perhaps with an anti-virus product installed on their machine, perhaps with a tool such as Fiddler or Wireshark to see what data the site is leaking about them, is tantamount. I'm saying if that's your use case, there's simply no reason to support it.
You also seem to have misunderstood what I said about the root certificates.
No, I understood perfectly, but it seems we're not communicating well.
Overall, I think it can be summed up as HPKP fully exists to help the 'destination server' control their security policy. Not allowing arbitrary 'loading' servers to control that policy was very much intentional.
@sleevi, I read the RFC7540 and it does not seem to agree with you. Specifically in the second paragraph it states:
The certificate presented by the server MUST satisfy any checks that the client would perform when forming a new TLS connection for the host in the URI.
This means you can't use a cert that would not be the one presented to you if you would have connected to the host in the URI in the first place.
Then in the third paragraph it actually explains the usage with wildcard certificates by stating that a cert can cover more than one URI, but nowhere does it say that a single URI can be covered by two different certs and that this is allowed to be cached. I believe this section might be the source of your confusion.
Then in the following two paragraphs it goes on to explain that if a server ends up getting a request to a URI that it is not authorized to serve it should fail the request with 421 (Misdirected Request). This makes very much sense: A server serving a.example.com with a wildcard cert. The cert covers b.example.com, however the server is not actually the right host. b.example.com is a different host and is using a different cert. We actually have this setup in our local network.
But even if this case was allowed, it would be irrelevant to this discussion because as I wrote in the first post the user must be able to pass not only one fingerprint but an array of acceptable fingerprints, and I added that for exactly the case where there is a group of certs that are all considered valid for the connection. The primary use for this is for the case where a cert expires or gets invalidated and the new cert's or the backup cert's fingerprint is used in addition, but can certainly also be used for load balancing or weird cases where routing or caching could end up using one cert or the other.
You've not established a use case where HPKP doesn't address, other than those that intentionally violate the priority of constituencies (anti-user) or invert control of security policy.
I thought I did that in my second post. Let me extract a specific case for you:
Please let me how HPKP is solving this problem.
This means you can't use a cert that would not be the one presented to you if you would have connected to the host in the URI in the first place.
This is simply an incorrect and wrong reading of the section. It doesn't mean the certificates need to be the same. It means exactly what was written - the certificate presented must satisfy the checks. This is not confusion on my part - I'm quite familiar with both the Firefox and Chromium implementation, having helped maintain the Firefox behaviour (which has further pooling independent of HTTP/2 and has for years) and reviewed and implemented the Chromium behaviour. As I indicated before, that's merely one example; if you want another, http://mxr.mozilla.org/nss/source/lib/ssl/sslnonce.c#276 goes back over 15 years of behaviour for NSS, reusing TLS session caches for independent hosts, provided the IP matches and the certificate satisfies any checks the client would perform.
But even if this case was allowed, it would be irrelevant to this discussion because as I wrote in the first post the user must be able to pass not only one fingerprint but an array of acceptable fingerprints,
No, you entirely missed my point. This is about the practical reality that an arbitrary third-party (e.g. NOT the server operator) is NOT in a position to determine what the appropriate pinset for a properly configured server is, because they're NOT in perfect knowledge of the set of certificates issued for a domain nor what is authorized. Certainly, this arbitrary third-party could make a best-effort stab at it, but all that does is arbitrarily and unnecessarily limit the server operator's ability to configure their server.
We're discussing about the real and practical flaws with such a proposal, and why it's been repeatedly and rightfully rejected (again, I encourage you to review past discussions of S-Links or the several predecessors).
the discovery server points the client to a server that provides the service it is looking for
Here's the fundamental problem, again restated. This discovery server is an arbitrary third-party entity, with no authorization to speak for the domain in question, nor necessarily any relationship to it, on a business or on a technical level. The failure of the discovery server to properly serve the keys either interferes with connections other than the one being made (this, again, is why we we're even discussing socket pools), or it requires additional pools (such that the only connection it affects is those it directs the user to).
Both of these are unnecessary complexity, but let's presume we introduced more pools and continue the thought experiment. This arbitrary discovery domain, having configure things such that it only affects the Fetch in question and with no global side-effects, has now limited the fetch() to either fail or succeed based on third-party-dictated security policy. While you may see this as a win, I see it as a loss. Imagine that 30% of a site's customers decided to employ this mechanism, because they found a bad example on StackOverflow that suggested they do it, and the example pinned the CA to "Foo CA". Now, "Foo CA" charges 10x of what "Bar CA" costs, and our hapless site only went with "Foo CA" because the old admin was naive and didn't realize there were cheaper options. The site goes and gets a cert from "Bar CA" - and suddenly sees their traffic - and revenue - drop 30%, because all these sites have wrong pins.
Now, the optimist in us might way "Gee, everything is working as intended; the site just needs to tell everyone to update their pins" - except the site has zero relationship with those arbitrary third-party sites (including the discovery server), and a benign change on the operator's part has now broken all of those links (thus defeating the primary goal of URLs to begin with). And recall, we only got to this point in our narrative because we handwaved away a number of problems that can't be handwaved away.
These concerns have been discussed - repeatedly - every time someone pokes a proposal like this. I can think of at least three past proposals. It's quite similar to the problems of subresource integrity (which, to be clear, I also have issues with), except on a whole new level, because the third-party does not have the same knowledge of the server operator and cannot simply observe as they can with hashes.
If that realistically is your threat model, then it should be on origins you control and operate - that's how you assure no MITM. This introduction problem is not merely one of technical ability, but in respecting the decentralized and disjoint nature of the web, and treating every party as potentially hostile. In such a model, such features are far more dangerous than good, and "Just give me a footgun, I promise not to shoot myself" is an argument that has repeatedly failed to play out as intended/promised.
@sleevi Your assumption that there is zero relationship with those sites is not true for all the examples I brought up so far. In the OpenID case the authentication endpoint is controlled by the same party that controls the OpenID configuration. In our distributed services case we are in a contractual relationship with our 3rd party service providers. We have a real threat model that you still failed to provide a solution for except "That's not how we do things here". I'm asking you to step back a bit and try to help me find a solution for this that is not a "footgun".
Your assumption that there is zero relationship with those sites is not true for all the examples I brought up so far.
It isn't an assumption - it's a statement about all the knowledge that a browser has. I haven't said it is impossible for there to be a business relationship - but that it is impossible for a browser to know such a relationship. The only way it can be modeled is by receiving the assent from the destination server to have it's policy dictated by the linking server, and that is fundamentally incompatible with your objectives (to avoid preloading and to avoid contacting the server).
We have a real threat model that you still failed to provide a solution for except "That's not how we do things here". I'm asking you to step back a bit and try to help me find a solution for this that is not a "footgun".
Indeed, because I fundamentally believe that your problem is not something that should be solved at this layer. Regardless of the validity of your threat model, the operational complexity and necessary tradeoffs are clear that the solution cannot and must not be at this layer.
I am not terribly interested in helping you engineer a solution, I will readily admit. I have neither the time, energy, nor interest. However, that blunt truth doesn't change the fact that what is proposed is a bad solution, and one that shouldn't be supported. My hope is that I have explained the reasons and concerns why, and that you might reconsider and explore alternatives that might mitigate these concerns, and if you find something, to discuss that. But my inability and unwillingness to provide free consulting wouldn't justify pursuing the proposed path.
Closing this per feedback from @sleevi.
I'd like to request support for certificate pinning.
Note that this is not the same as HSTS/HPKP although it seems to be related.
I see two use cases that I believe are both useful:
Global
The first one is to have a global hash of valid domains to valid fingerprints. All HTTPS requests would look up the domain in the hash and match the cert (chain) to the valid fingerprints. Unexpected certificates would fail the request.
A request could have a flag that indicates whether it wants to opt out of this or weather the request should fail or instead just continue with a flag set. I do think that it should fail by default because this would allow to add certificate pinning to an application without changing all the fetch calls.
Example:
Per call
The second case is a per-call option. An additional parameter in the option hash would give valid expected fingerprints for this request. If the server cert does not match one of the passed fingerprints the request would fail (or be flagged).
Example:
This could be combined with the global list of fingerprints with the same flag as in the global case, so the possible cases would be:
Since HPKP only supports sha256 certificate fingerprints (for now) I think it should at least support that. Sha1 might be an option as well. Just like HSTS the parameters or functions should have different names according to the fingerprint types they accept.
I'm not sure about the specific names for the functions and parameters, any comments or suggestions are welcome.
As an alternative to this I could also see the server cert chain including the public keys and/or its fingerprints be exposed in the response which would allow the client to do cert pinning by itself. This could potentially be more useful for other applications (than cert pinning) but would also make cert pinning more tedious.