Consider using system trust stores by default in 3.0.0.

[Lukasa]

It's been raised repeatedly, mostly by people using Linux systems, that it's annoying that requests doesn't use the system trust store and instead uses the one that certifi ships. This is an understandable position. I have some personal attachment to the certifi approach, but the other side of that argument definitely has a reasonable position too. For this reason, I'd like us to look into whether we should use the system trust store by default, and make certifi's bundle a fallback option.

I have some caveats here:

1. If we move to the system trust store, we must do so on all platforms: Linux must not be its own special snowflake.
2. We must have broad-based support for Linux and Windows.
3. We must be able to fall back to certifi cleanly.

Right now it seems like the best route to achieving this would be to use certitude. This currently has support for dynamically generating the cert bundle OpenSSL needs directly from the system keychain on OS X. If we added Linux and Windows support to that library, we may have the opportunity to switch to using certitude.

Given @kennethreitz's bundling policy, we probably cannot unconditionally switch to certitude, because certitude depends on cryptography (at least on OS X). However, certitude could take the current privileged position that certifi takes, or be a higher priority than certifi, as an optional dependency that is used if present on the system.

Thoughts? This is currently a RFC, so please comment if you have opinions. /cc @sigmavirus24 @alex @kennethreitz @dstufft @glyph @reaperhulk @morganfainberg

[dstufft]

I think the system trust stores (or not) essentially boils down to whether you want requests to act the same across platforms, or whether you want it to act in line with the platform it is running on. I do not think that either of these options are wrong (or right), just different trade offs.

I think that it's not as simple on Windows as it is on Linux or OSX (although @tiran might have a better idea). I think that Windows doesn't ship with all of the certificates available and you have to do something (use WinHTTP?) to get it to download any additional certificates on demand. I think that means that a brand new Windows install, if you attempt to dump the certificate store will be missing a great many certificates.

On Linux, you still have the problem that there isn't one single set location for the certificate files, the best you can do is try to heuristically guess at where it might be. This gets better on Python 2.7.9+ and Python 3.4+ since you can use ssl.get_default_verify_paths() to get what the default paths are, but you can't rely on that unless you drop 2.6, 2.7.8, and 3.3. In pip we attempt to discover the location of the system trust store (just by looping over some common file locations) and if we can't find it we fall back to certifi, and one problem that has come up is that sometimes we'll find a location, but it's an old outdated copy that isn't being updated by anything. People then get really confused because it works in their browser, it works with requests, but it doesn't in pip.

I assume the fall back to certifi ensures that things will still work correctly on platforms that either don't ship certificates at all, or don't ship them by default and they aren't installed? If so, that's another possible niggle here that you'd want to think about. Some platforms, like say FreeBSD, don't ship them by default at all. So it's possible that people will have a requests using thing running just fine without the FreeBSD certificates installed, and they then install them (explicitly or implicitly) and suddenly they trust something different and the behavior of the program changes.

Anyways, the desire seems reasonable to me and, if all of the little niggles get worked out, it really just comes down to a question of if requests wants to fall on the side of "fitting in" with a particular platform, or if it wants to prefer cross platform uniformity.

[glyph]

I think the system trust stores (or not) essentially boils down to whether you want requests to act the same across platforms, or whether you want it to act in line with the platform it is running on. I do not think that either of these options are wrong (or right), just different trade offs.

I wouldn't say that either option is completely wrong, but I do think that using the platform trust store is significantly right-er, due to the availability of tooling to adjust trust roots on the platform and the relative unavailability of any such tooling for requests or certifi. If you go into Keychain Access to add an anchor (or the Windows equivalent) nothing about requests makes it seem like it would be special, that it would be using a different set of trust roots than you had already configured for everything else.

[dstufft]

It depends if your audience are people who are familiar with a particular platform or not. I have no idea how to manage the trust store on Windows but I know how to manage the trust store for requests because requests currently chooses being internally consistent cross platform over being externally consistent with any particular platform.

IOW this change makes it easier for people who are familiar with the system the software is running on at the cost of people who are not.

Sent from my iPhone

On Jan 11, 2016, at 4:29 PM, Glyph notifications@github.com wrote:

I wouldn't say that either option is completely wrong, but I do think that using the platform trust store is significantly right-er

[glyph]

IOW this change makes it easier for people who are familiar with the system the software is running on at the cost of people who are not.

In the abstract, I disagree. Of course, we may have distribution or build toolchain issues which make people have to care about this fact, but if it works properly (pip installs without argument, doesn't require C compiler shenanigans for the end user) then what is the penalty to people who are not familiar with the platform?

[dstufft]

It forces them to learn the differences of every platform they are running on.

Sent from my iPhone

On Jan 11, 2016, at 4:59 PM, Glyph notifications@github.com wrote:

what is the penalty to people who are not familiar with the platform?

[tempusfrangit]

So I am in favour of using the system store in general because in most cases [outside of dev] if you're relying on requests or a similar library you're going to expect it to work similar to the rest of the system/tooling. Asking someone to learn the tooling for the system they are deploying on is not unreasonable. If an application uses requests and needs to trust a specific cert (end-user story here), it is usual that it'll handle the installation for that platform at install time (aka OS X or Windows).

From a developer perspective, it becomes a little more difficult but still not insurmountable as long as we approach this in a way that the developer has clear methods to continue with the same behaviour as today.

As discussed in IRC, perhaps the easiest method to ensure sanity is to really finish up and polish certitude(? was this the tool discussed?) so we can encapsulate the platform/system-specifics clearly and try and ensure requests logic is the same on all of the platforms.

[glyph]

It forces them to learn the differences of every platform they are running on.

How so? By default, it ought to get the trust roots you expect on every platform. It's not like I need to learn new and exciting things when I launch Safari vs. IE just to type https://...

[glyph]

I think that it's not as simple on Windows as it is on Linux or OSX (although @tiran might have a better idea). I think that Windows doesn't ship with all of the certificates available and you have to do something (use WinHTTP?) to get it to download any additional certificates on demand. I think that means that a brand new Windows install, if you attempt to dump the certificate store will be missing a great many certificates.

You are right about this. I have verified it on my nearly-pristine Windows VM; in a Python prompt, I do:

>>> import wincertstore
>>> print(len(list(wincertstore.CertSystemStore("ROOT"))))

and get "21". Visit some HTTPS websites, up-arrow/enter in the python interpreter, and now I get "23".

[glyph]

There's some technical documentation here:

https://technet.microsoft.com/en-us/library/bb457160.aspx

[glyph]

And some more explanation here:

http://unmitigatedrisk.com/?p=259

[glyph]

Frustratingly, I can't find an API that just tells it to grab the certificate store; it seems that verifying a certificate chain that you don't have the root to is the only way that it adds certificates, and it adds them one at a time as necessary. It baffles me that Microsoft seems to consider storage for certificates a scarce resource.

[glyph]

After hours of scouring MSDN, I give up. Hopefully someone else can answer this question: https://stackoverflow.com/questions/34732586/is-there-an-api-to-pre-retrieve-the-list-of-trusted-root-certificates-on-windows

[dstufft]

In my experience with pip, which attempts to discover the system store and if it can't find it falls back to a bundled copy, I have had to learn how the system store works on platforms that I have no intention on ever running. This is for a fairly simple method of detection (look for file systems) but it absolutely ends up that way. The simple fact is, in my experience most people have absolutely no idea how their system manages a trust store (and if it manages a trust store or not).

Sent from my iPhone

On Jan 11, 2016, at 5:22 PM, Glyph notifications@github.com wrote:

How so? By default, it ought to get the trust roots you expect on every platform. It's not like I need to learn new and exciting things when I launch Safari vs. IE just to type https://...

[dstufft]

Look for files on the system*

[dstufft]

FWIW, If I could prevent downstream redistributors from forcing pip to use the system store, I would revert the change to look in system locations immediately and only ever use a bundled copy. The UX of that tends to be so much nicer, the only reason we started to trust the system trust stores is because redistributors do patch pip to use the system trust store, so you end up in a situation where people get different trust stores based on where their copy of pip came from (which is likely also a concern for requests).

As an additional datapoint, if I remember correctly, the browsers which are not shipped with the OS tend to not use the system trust store either. According to Chrome's Root Certificate Policy they will use the system trust store on OSX and on Windows but they won't use it on Linux. I believe that even where Chrome does use the system store, they still layer their own trust ontop of that to allow them to blacklist certificates if need be. I assume this capability is in place because they do not wholly trust the OS trust stores to remove compromised certificates. If I recall correctly, Firefox does not use the system trust store at all on any OS.

Another question is what even is the "root trust store" on a Linux. The closest that you can get is wherever the system provided OpenSSL (assuming they even provide OpenSSL) is configured to point to. However AFAIK there is no way to determine if you're using a system provided OpenSSL or a copy that someone installed (perhaps via Anaconda?), the additional copy may have stale certificates or no certificates available. If it's stale certificates, then you've successfully lowered the security of requests user's by attempting to follow the system trust store. If it's an empty trust store, how do you determine the difference between "empty because I trust nothing" and "empty because my copy of OpenSSL isn't shipping them" or do I have to manage the certificates I trust using my OS, unless I want to trust nothing then I have to manage the certificates I trust using requests?

I've also found that talks about trying to use the platform certificate trust store on Linux (see here). I've not personally verified the information in this article, however it makes me feel very wary about trying to make using the system trust store anything but an exercise in frustration.

[kennethreitz]

I think our current approach is the correct one, considering who Requests was built for.

That being said, it wouldn't hurt to add more documentation/functionality around using system certs for "advanced" users.

[glyph]

I think talking about configurability is maybe a red herring. It's useful – necessary even – in certain circumstances, but users who know they need that can usually figure it out.

The more significant issue is that trust root database updates, especially for end-user client software, are both infrequent and extremely important to do in a timely manner. certifi has no mechanism for automatically updating. Not only will it not come down in a system update, it can't even be done globally; you have to do it once per Python environment (virtualenv, install, home directory, etc, where it's installed).

[tempusfrangit]

@glyph I think you may have hit the nail on the head there. That is a significant reason (and addresses the other issues outlined) to use a more centralized location for the cert store.

[dstufft]

The flip side is that you have trust stores like Debian which trusted CACert for a long time, and still trusts SPI even though neither of those have gone through any sort of real audit that you'd expect for a trusted CA.

[glyph]

The flip side is that you have trust stores like Debian which trusted CACert for a long time, and still trusts SPI even though neither of those have gone through any sort of real audit that you'd expect for a trusted CA.

As I understand it, Microsoft trusts their own CA, too, and Apple trusts theirs. SPI is just Debian's version of that, isn't it?

[dstufft]

SPI isn't run by Debian, it's a third party organization similar to that of the Software Freedom Conservancy that Debian happens to be a member of. It'd be more like Microsoft and Apple trusting the CA of their datacenter just because they happened to use them as their data center. In addition, I'm pretty sure that Apple and Microsoft have both passed a WebTrust audit for their root CAs.

[dstufft]

To be fair to Debian, I think the current plan is to stop using SPI certificates for their infrastructure and switch to more generally trusted certificates and then stop including SPI and switch to using just the Mozilla bundle without any additions.

That being said, is it even true that they are shipping updates to them? Looking at packages.debian.org for the ca-certificates packages it shows that the versions there are:

• squeeze (oldoldstable): 20090814+nmu3squeeze1
• wheezy (oldstable): 20130119+deb7u1
• jessie (stable): 20141019
• stretch (testing): 20160104
• sid (unstable): 20160104

I haven't looked at the actual contents of these packages, but the version numbers lead me to believe that they are not infact keeping the ca-certificates package up to date.

In addition to that, looking at the open bugs for ca-certificates there are bugs like #721976 which means that the ca-certificates store includes roots which are not valid for validating servers and are only valid for other topics (like email) which means you can't actually use the current ca-certificate package without massaging it to remove those certificates yourself.

Another issue #808600 has Comodo requesting a removal of a particular root that they no longer consider to be inscope for the CAB Forum's Baseline Requirements. That has been removed from testing and sid, but has not been removed from jessie, wheezy, or squeeze. The maintainers of ca-certificates claim they'll be requesting an upload to jessie and wheezy, but not to squeeze (which still has LTS suport).

That's just from spending a little bit of time looking at one, fairly popular, distribution. I imagine the long tail of the issues with the system provided bundle gets worse the further away from the popular distributions you get. It's not clear to me that it's a reasonable assumption that the certificates included in any random OS are going to be properly maintained.

The other elephant in the room is we're just assuming that because an update is available to their ca-certificates package that someone is going to have pulled it in. As far as I know, most (if not all?) of the Linux systems do not automatically update by default and require configuration to start to do so. There is likely to be a bigger chance of this with Docker in the picture. On the flip side, I think people generally try to update to the latest versions of their dependencies when working on a project.

[Lukasa]

Debian's cert bundle is almost certainly like ubuntu's, which does not update to the Mozilla bundle that removed 1024-bit roots in order to avoid the pain like that which hit certifi. All of the out-of-date cert bundles are OpenSSL pre 1.0.2, which means they cannot correctly build the cert chain to a cross-signed root without having the 1024-bit cross-signing root still present. I suspect that's the real concern there.

[dstufft]

Maybe they should stop shipping an OpenSSL that can't correctly validate certificate chains.

[alex]

I reached out to Kurt Roebx about backporting the fixes for that, he said it was a thing he was looking at doing, I have no clue what the timeline is.

On Wed, Jan 13, 2016 at 7:27 AM, Donald Stufft notifications@github.com wrote:

Maybe they should stop shipping an OpenSSL that can't correctly validate certificate chains.

— Reply to this email directly or view it on GitHub https://github.com/kennethreitz/requests/issues/2966#issuecomment-171276369 .

"I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero GPG Key fingerprint: 125F 5C67 DFE9 4084

[sigmavirus24]

So, what if we made the behaviour of looking at the system CA bundle an extra, e.g., pip install requests[system_ca] where it would ship without our certificate bundle and instead use certifi. This allows us to keep on "just work" ing for our default user base while supporting the people who need to use the system bundle.

[dstufft]

Seems reasonable.

[alex]

I would start with adding an API for it, before worrying about how to enable it via installation, you need requests.get(url, verify=<something>) to exist first.

On Wed, Jan 13, 2016 at 7:41 AM, Ian Cordasco notifications@github.com wrote:

So, what if we made the behaviour of looking at the system CA bundle an extra, e.g., pip install requests[system_ca] where it would ship without our certificate bundle and instead use certifi. This allows us to keep on "just work" ing for our default user base while supporting the people who need to use the system bundle.

— Reply to this email directly or view it on GitHub https://github.com/kennethreitz/requests/issues/2966#issuecomment-171279745 .

"I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero GPG Key fingerprint: 125F 5C67 DFE9 4084

[tiran]

Debian / Ubuntu have more issues. They ignore CKT_NSS_MUST_VERIFY_TRUST flag and throw all trust anchors in one PEM file. The flag overrides extended key usage (EKU) for root certs, e.g. only trust a root cert for S/MIME but not for TLS server auth.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=721976 https://bugs.launchpad.net/ubuntu/+source/ca-certificates/+bug/1207004

Apple, Microsoft and Red Hat/Fedora have means to disable certs for a purpose. Red Hat only puts trust anchors for TLS server auth in the default cert bundle. With FreeDesktop.org's p11kit and PKCS#11 bindings the policies can even be modified by user. This doesn't work for OpenSSL yet because OpenSSL uses a static PEM file instead of PKCS#11 to fetch trust anchors.

@glyph Does MS trust their own cert for all EKUs or just for some EKUs like code signing?

[sigmavirus24]

@alex would something like

import requests
from requests import certs

requests.get(url, verify=certs.system_bundle_where())

Be a sufficiently good API?

[Lukasa]

Note that that is already the API certitude provides.

[glyph]

SPI isn't run by Debian, it's a third party organization similar to that of the Software Freedom Conservancy that Debian happens to be a member of.

This is a bit of a tangent, but; no, not quite.

“SPI was originally created to allow the Debian Project to accept donations. It now acts as a fiscal sponsor to many free and open source projects.”

[glyph]

I'm pretty sure that Apple and Microsoft have both passed a WebTrust audit for their root CAs.

I checked and this is definitely true for Apple and seems to be true for Microsoft. So, yeah, SPI isn't cutting the mustard here.

[dstufft]

SPI was originally created to allow the Debian Project to accept donations. It now acts as a fiscal sponsor to many free and open source projects.

originally is an important word here, it started as something ran by Debian, but it's now fully independent AFAIK.

[glyph]

To sum up the argument thus far, I think it's:

<glyph> it's the OS's job to update certificates
<dstufft> debian does a bad job

I don't think these statements really conflict. And I don't disagree, at all: Debian is being somewhat negligent in their treatment of root CAs, and that could compromise Debian users' security. Somebody should be yelling at them about that. But from a threat modeling perspective, if we assume that Debian is including a bad / compromised CA, that means the get-pip.py you got was subverted in transit anyway, because curl is using the system CA list too. I'd still rather have a bad CA cert list from my OS than 50 bad CA cert lists from applications, since the OS is a lot easier to update centrally. This applies even to Docker containers: rebuilding an image with an upgrade line at the top should get the latest of everything, including trust roots.

[dstufft]

The other side of that is, if the people who should do the job did it and we were happy with that, we'd just let OpenSSL pick our default cipher strings too.

[glyph]

The other side of that is, if the people who should do the job did it and we were happy with that, we'd just let OpenSSL pick our default cipher strings too.

It's a fair cop.

[mithrandi]

First, a couple of points:

1) The SPI CA is no longer in ca-certificates as of version 20151214.

2) The single PEM CA bundle is only used by software that is unable to use a directory of certificates. (Although the issue of conflating different types of trust root is real)

I think the most interesting thing about the ca-certificates package is that it allows an administrator to enable/disable certificates if they so choose. Of course, an uninformed user isn't going to bother with this, but if you are interested / informed, and everything uses the system CA configuration, you have a central place to view / manage things; while if everything bypasses this and ships their own bundles, it's basically impossible to keep track of everything no matter what type of user you are.

(This is already an issue since the upstream versions of Firefox, Chrome, etc. all use their own certificate stores...)

[smiley]

I think true native support (i.e.: mapping "is this certificate okay?" requests to OS APIs) would be preferable. This would help with the problem of automatically-updating certificate stores on Windows, along with supporting enterprise certificate authorities. Here I had to manually import our root certificates into the certificate bundle, and if it gets reissued across the domain, we'd need to repeat the process. (Which could get lost over time)

Is that possible? Or is certificate verification ultimately handled by Python/urllib, and Requests merely supplies the bundle?

(This is already an issue since the upstream versions of Firefox, Chrome, etc. all use their own certificate stores...)

I think Firefox is the only one that has its own store, Chrome (at least on Windows) uses the system store.

[Lukasa]

I think true native support (i.e.: mapping "is this certificate okay?" requests to OS APIs) would be preferable.

That is very nearly impossible.

The issue there is that there are three major platforms (Windows, OS X, other), each of which ships with their own "OS-default" library for validating TLS certificate chains. For obvious reasons, these libraries all also believe they are in charge of doing the TLS record building and handshaking and all the other fun stuff. That means that integrating with those libraries requires writing three separate wrapper libraries, each of which may fail in exciting and different ways.

Getting those to function together correctly would be extremely difficult. Even if we could do that, I rolled all the "other" platforms together because they'd provide OpenSSL, but in fact many would either not provide that (e.g. BSD providing LibreSSL) or the user would prefer we used a different library (GnuTLS, PolarSSL, NSS).

It should be noted that, as far as I can tell, literally the only piece of web software that attempts to do this is curl: not even web browsers try. Firefox bundles its own TLS library (NSS), as does Chrome (BoringSSL), while Safari and Edge do not run across multiple platforms and so can stick their hooks into the OS-provided libraries. Even curl does not appear to hook into Windows' default store.

Currently the assumption is that any change here would simply adjust to use a PEM bundle provided by the OS. On *nix, this should be in one of a few well-known locations, and certitude could simply proxy to it. On OS X, we can extract the certificates from the keychain and write them into a PEM file: this is what certitude does today. On Windows we could in principle do this, but there's some trickiness about the fact that Windows dynamically fetches some root certificates (a truly mind-boggling design decision).

[alex]

Chrome uses the platform trust stores, including their provided functions for checking validity.

On Mon, Jan 18, 2016 at 6:15 AM, Cory Benfield notifications@github.com wrote:

I think true native support (i.e.: mapping "is this certificate okay?" requests to OS APIs) would be preferable.

That is very nearly impossible.

The issue there is that there are three major platforms (Windows, OS X, other), each of which ships with their own "OS-default" library for validating TLS certificate chains. For obvious reasons, these libraries all also believe they are in charge of doing the TLS record building and handshaking and all the other fun stuff. That means that integrating with those libraries requires writing three separate wrapper libraries, each of which may fail in exciting and different ways.

Getting those to function together correctly would be extremely difficult. Even if we could do that, I rolled all the "other" platforms together because they'd provide OpenSSL, but in fact many would either not provide that (e.g. BSD providing LibreSSL) or the user would prefer we used a different library (GnuTLS, PolarSSL, NSS).

It should be noted that, as far as I can tell, literally the only piece of web software that attempts to do this is curl https://github.com/bagder/curl/tree/master/lib/vtls: not even web browsers try. Firefox bundles its own TLS library (NSS), as does Chrome (BoringSSL), while Safari and Edge do not run across multiple platforms and so can stick their hooks into the OS-provided libraries. Even curl does not appear to hook into Windows' default store.

Currently the assumption is that any change here would simply adjust to use a PEM bundle provided by the OS. On nix, this should be in one of a few well-known locations, and certitude could simply proxy to it. On OS X, we can extract the certificates from the keychain and write them into a PEM file: this is what certitude does today. On Windows we could *in principle do this, but there's some trickiness about the fact that Windows dynamically fetches some root certificates (a truly mind-boggling design decision).

— Reply to this email directly or view it on GitHub https://github.com/kennethreitz/requests/issues/2966#issuecomment-172500959 .

"I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero GPG Key fingerprint: 125F 5C67 DFE9 4084

[dstufft]

According to https://www.chromium.org/Home/chromium-security/root-ca-policy Chrome only uses the platform trust store on Windows and OSX.

[dstufft]

Because an issue in pip made me think about this again!

On *nix, this should be in one of a few well-known locations, and certitude could simply proxy to it.

I think this is wrong. It's the method currently in use by pip but we're moving away from it because just because a file exists in one of the "well known locations" doesn't mean that it's the right file for that system and assuming it is opens people up to using a stale CA bundle that isn't being managed by anything. If you're going to do anything, it should be to utilize the OpenSSL APIs to get the default OpenSSL ca bundle.

That brings me to the next point in this, currently the OpenSSL APIs to get the default CA bundle are broken on Debian. Attempting to query it gives you a location that doesn't exist because it is the wrong location. This is #805646 which results in (on Debian Stable):

$ python -c "import ssl; print(ssl.get_default_verify_paths())"
DefaultVerifyPaths(cafile=None, capath='/usr/lib/ssl/certs', openssl_cafile_env='SSL_CERT_FILE', openssl_cafile='/usr/lib/ssl/cert.pem', openssl_capath_env='SSL_CERT_DIR', openssl_capath='/usr/lib/ssl/certs')
$ python3 -c "import ssl; print(ssl.get_default_verify_paths())"
DefaultVerifyPaths(cafile=None, capath='/usr/lib/ssl/certs', openssl_cafile_env='SSL_CERT_FILE', openssl_cafile='/usr/lib/ssl/cert.pem', openssl_capath_env='SSL_CERT_DIR', openssl_capath='/usr/lib/ssl/certs')

The openssl_cafile should be /etc/ssl/certs/ca-certificates.crt (I think). I am unsure what the full impact of this actually is in terms of what versions of Debian and if it extends to any of the Debian derivatives like Ubuntu.

[glyph]

Echoing the Chrome policy that @dstufft referenced above, I think that maybe the right thing to do here is (for now) to trust the Windows and OS X trust stores (inasmuch as we can) since Microsoft and Apple have demonstrated a baseline level of trustworthiness in their management of certificates.

It seems like neither Debian nor Red Hat is similarly responsible, at least, at this time.

@dstufft asks: what is different about trusting the OS's cert stores than trusting the OS to properly configure OpenSSL? In the Microsoft and Apple case, they knew better than to ship OpenSSL, because it's not actually a proper transport security solution: it's part of a construction kit for building your own transport security. They built their own, more complete TLS implementations, which have interfaces that let you do things like select trust roots in a sensible way. In a perfect world, IMHO, we'd be using cryptography.tls, which would actually back-end to SChannel or Secure Transport on those platforms, and bundle its own OpenSSL only for linux. Given that we're a long way away from that world (just in terms of the amount of work it would take), bundling OpenSSL for the protocol implementation on all platforms but trusting the platform trust store on those platforms which seem trustworthy, which hopefully eventually will be all of them.

[dstufft]

@glyph well, except AFAIK Windows doesn't provide a way to either enumerate the SSL Certificates in a way that you actually get them all, not just the ones you've seen. Perhaps you an use SChannel for that (is that a thing you can do? last time I saw this someone was trying to replace urllib3 with WinHTTP to get requests to trust the windows trust store), but then if you're relying on SChannel you're also relying on Microsoft for some other things which might be undesirable. It'd mean that there's no more SNI for Windows XP users, no ECDHE+AESGCM for anyone not on Windows 10,

As far as I know, the only way to get ideal security right now on any platform in a way that isn't tied to the absolute latest versions of that platform is to bring your own TLS library and your own certificates.

[dstufft]

Honestly, the single greatest thing that could probably be done to increase security is for cryptography to start shipping statically compiled wheels on Linux too so you can get a modern TLS implementation on all the platforms.

[mwcampbell]

I think the best solution is to use each platform's native TLS implementation. On Windows, that's SChannel; on Mac, that's SecureTransport; on Fedora, that appears to be NSS; elsewhere, it's probably OpenSSL. That would make the host platform responsible for certificate verification, including dynamic downloading of new CA certs where necessary (on Windows). Also, on platforms where OpenSSL is not available, let alone the native solution (Windows, Android via the small set of libraries exported by the NDK), it would be great if application developers didn't have to bundle their own copy of a crypto/TLS library and keep it up to date.

Of course, implementing an abstraction over all of these libraries is a lot of work. libcurl has already done this in its vtls module. But that's currently internal to libcurl, and I assume that modifying requests to use libcurl instead of urllib3 is a bigger change than anyone here wants to make. However, the curl developers previously did some work on exporting vtls as a separate library; maybe we could convince them to revive that project.

[glyph]

It's bad enough that we have to use OpenSSL (or the platform SSL bindings); cURL is written in C, and like all software written in C, we periodically discover, much to our surprise, that it contains remote code execution and crashing vulnerabilities. So personally I'd rather avoid it here.

[Lukasa]

Ok, so let's think about this a different way slightly. Right now the discussion has focused on "where do we get the certificates", but that's not really a sufficiently detailed question because it just doesn't work on Windows.

So let's rephrase the question. Should be be using OpenSSL to verify the certificate on Windows/OS X? With PyOpenSSL, at the very least, it should be possible for us to do what Chrome does and intervene on the verify_callback and pass that certificate to the OS-appropriate verification functions.

Chrome has Windows-specific code for this and OS X-specific code for this. It is in principle possible to pull this functionality into cryptography and then, on Mac and Windows, essentially throw the verification out to the OS-specific functions rather than using OpenSSL.

My reservation about this approach is that the Python standard library does not make this approach available to us. However, we could potentially have the following approach:

1. Check if cryptography and PyOpenSSL are installed. If they are, and the user is on Windows or OS X, use the system cert bundle as exposed by the system's own APIs.
2. (Maybe, maybe not) Otherwise, if certitude is installed, use that cert bundle.
3. Otherwise, if certifi is installed, use that cert bundle.
4. Otherwise, use the vendored bundle.

Thoughts?

[dstufft]

Just to further hammer home my point:

• Homebrew on OSX ships with a broken CAPath because it's empty but the directory exists.
• Debian ships with a broken CAFile which doesn't exist.
• CentOS and Fedora ship with a broken CAPath which is just some symlinks and doesn't work.

Pip is giving up on trying to trust the system stores, systems can't be trusted to ship things in a state that isn't broken. See https://github.com/pypa/pip/issues/3415.