Client Certificates w/Passphrases?

[tdussa]

Hi,

client certificates can be specified with the "cert" parameter. If I pass an encrypted client certificate, the underlying OpenSSL call will query for the corresponding passphrase, but that is not really a feasible way of handling this for a larger session with multiple calls, because the password is obviously not cached in any way.

Is there any way to pass the passphrase to a connection with API calls so it is then passed on to OpenSSL?

Cheers, Toby.

[sigmavirus24]

Hi @tdussa,

In the future, please ask questions on StackOverflow. Quoting from the docs.

cert – (optional) if String, path to ssl client cert file (.pem). If Tuple, (‘cert’, ‘key’) pair.

So if you have the key file, you can use that. @t-8ch can correct me if I'm wrong, but I don't believe we (or urllib3) support sending a passphrase. I think the lack of support is specifically a limitation of the way the SSL module loads verification data.

[tdussa]

Hi,

THX for your quick response. Admittedly, I was hoping that this would just be a question, but I am afraid that it will actually be a feature request if I read you correctly.

So I have tried to read through the relevant portions of the request and urllib3 code, and I had hoped that I had overlooked some obvious way of passing a passphrase to pyOpenSSL.

Actually, the underlying SSL code DOES support encrypted (password-protected) client certificates; the relevant function that is called by urllib3 is not load_verify_locations but load_cert_chain (which I also think is badly named). So pyOpenSSL does support what I'm looking for, but I haven't been able to figure out a way of actually passing a password argument through requests/urllib3 to pyOpenSSL. I have started thinking about how to best patch this into both requests and urllib3, but the actual code path is not that obvious to me right now.

THX & Cheers, Toby.

[sigmavirus24]

So the appropriate way to do this, would be for me to pick work back up on https://github.com/shazow/urllib3/pull/507 since it would provide the API you're looking for. You would create a SSLContext object and call load_cert_chain yourself with the password and give that to us to use.

[tdussa]

Sounds about right, yeah. So if that would be possible, that'd be awesome.

THX & Cheers, Toby.

[tdussa]

Is there any estimate as to when this feature might be added?

[Lukasa]

@tdussa Not at this time I'm afraid. =(

[tdussa]

Bummer. :( Looking forward to it. ;-)

[traut]

Waiting for this feature as well. Any estimates?

[Lukasa]

Nope. There's a big chunk of work to be done for this.

[traut]

Is it not even in the next milestone?

[Lukasa]

Not currently, no. We have a finite amount of resources to spend, and the development team is stretched pretty thin across a wide number of projects.

[traut]

any advice on how to get to urllib3's context.load_cert_chain from requests.Session or prepared request ? I'm looking for a workaround

[Lukasa]

@traut There isn't a good one, really. You can attempt to use the TransportAdapter's init_poolmanager method to pass objects into urllib3.

[traut]

any progress on this issue?

[Lukasa]

@traut Yes. We're a few releases away from users being able to use TransportAdapters to provide SSLContext objects to urllib3, which will resolve this issue.

[traut]

nice! thanks for the update, @Lukasa

[aiguofer]

Looks like this is now possible. Here's an example of how I made this work:

https://gist.github.com/aiguofer/1eb881ccf199d4aaa2097d87f93ace6a

[amiralia]

My question is very related, but not quite the same. Is there a way to pass the unencrypted certificates and key as raw_bytes or as file objects, rather than file paths? Since for security reasons, one may not want to store the certificates on disk.

[Lukasa]

@amiralia Right now the answer is "not easily". The Python standard library ssl module exposes no way to load client certs/keys from bytes: only from files. This is despite the fact that OpenSSL itself does provide these tools.

One way around this is to ensure you're using Requests PyOpenSSL support. If you do that, you can get an SSLContext object that uses PyOpenSSL instead by calling requests.packages.urllib3.util.ssl_.create_urllib3_context(). That will let you call PyOpenSSL methods like use_certificate, which will work with files loaded from memory.

Unfortunately, until the stdlib changes its API, that is the only option.

[amiralia]

I tried that, but even with using the following code for injection, and then creating the context as described, the SSL_Context object didn't have use_certificate or use_privatekey methods. import urllib3.contrib.pyopenssl urllib3.contrib.pyopenssl.inject_into_urllib3()

[Lukasa]

The object itself does not, but it has a ctx object on it that does. You'll need to look at the actual code in that module to see how it works.

[amiralia]

Ok, I got it to work, but there was a very weird bug where the injection overwrote the variables under requests.packages.urllib3.util, but not requests.packages.urllib3.util.ssl_, including the important requests.packages.urllib3.util.ssl_.SSLContext. I resolved it by doing a manual monkey patch in my code of the remaining variables.

[amiralia]

Thanks again everyone!

[bedge]

Here's an alternative. Re-encode the cert to not require a passphrase.

Run this script passing the .p12 as an argument. It'll prompt for the passphrase and generate a .pem that doesn't need one. Then it asks for a new passphrase (twice, for validation) that's used to build a new .p12. You can leave it blank and the result is a .p12 and .pem that don't require passphrases.

#!/bin/bash -e
np_pem=${1/.p12/_np.pem}
np_p12=${np_pem/pem/p12}
openssl pkcs12 -in $1 -nodes -out $np_pem
echo "Press <CR> twice here for empty passphrase"
openssl pkcs12 -export -in $np_pem  -out $np_p12

[amiralia]

Thanks bedge! That's good to know. But for our security requirements, we're mandated by the company to keep passphrases.

[cben]

workaround idea: maybe it's possible to re-encode into a temp file that never hits the disk, yet can be accessed as named file through /dev/fd/N? tempfile.TemporaryFile(), unix only.
Security depends on O_TMPFILE support or unpredictability of file name, and on non-tmpfs filesystems on question whether contents for an unnamed inode might still be flushed out to disk (though not accessible from any directory). See also https://unix.stackexchange.com/questions/74497/single-process-accessible-temporary-file

EDIT: note that other processes by same user can see and read the temp file via a path like /proc/20782/fd/3. (The symlink reads as broken 3 -> '/tmp/#345829 (deleted)' but opening does work. Rescuing deleted but still open files is a deliberate use case of /proc/NN/fd...)
There are of course other attack vectors from processes by same user, e.g. ptrace...

[qfayet]

Here is a solution that I found by taking inspiration from this article:
Creating a Python requests session using a passphrase protected Client side Cert

import ssl
import requests
from requests.adapters import HTTPAdapter

class SSLAdapter(HTTPAdapter):

    def __init__(self, certfile, password):
        self.context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
        self.context.load_cert_chain(certfile=certfile, password=password)
        super().__init__()

    def init_poolmanager(self, *args, **kwargs):
        kwargs['ssl_context'] = self.context
        return super().init_poolmanager(*args, **kwargs)

session = requests.session()
session.mount('https://my_protected_site.com', SSLAdapter('my_certificate.crt', 'my_passphrase'))

[amiralia]

Yes, that's what I ended up using as well those years ago. It should be doable for requests to wrap that logic in a function call for get_secure_session? Amirali Abdullah

Software Engineering

http://www.facebook.com/Qualtrics http://www.twitter.com/Qualtrics http://www.instagram.com/qualtrics http://www.linkedin.com/company/qualtrics

On Sun, May 19, 2019 at 8:31 AM qfayet notifications@github.com wrote:

Here is a solution that I found by taking inspiration from this article: https://gist.github.com/aiguofer/1eb881ccf199d4aaa2097d87f93ace6a

import ssl import requests from requests.adapters import HTTPAdapter

class SSLAdapter(HTTPAdapter):

def init_poolmanager(self, *args, **kwargs):
    context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
    context.load_cert_chain(certfile='my_certificate.crt',
                            password='my_password')
    kwargs['ssl_context'] = context
    return super().init_poolmanager(*args, **kwargs)

session = requests.session() session.mount('https://my_protected_site.com', SSLAdapter())

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/kennethreitz/requests/issues/2519?email_source=notifications&email_token=AE4W6UK3AZ7QPKBDWM6M7QDPWFQC7A5CNFSM4A64TK5KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODVXDMJQ#issuecomment-493762086, or mute the thread https://github.com/notifications/unsubscribe-auth/AE4W6UNZRK5TI7Z2PI46DRDPWFQC7ANCNFSM4A64TK5A .

[fdcds]

I believe this is a duplicate of #1573?

[yohonet]

AFAIK, it's still not supported today (at least according to documentation). Looks like it has been requested for a long time. Is there any plan for this ?

[tbennett6421]

Would you be willing to accept pull requests that implement this?

[aradkdj]

Today I stumbled upon this. If it helps, urllib3 supports a key_password keyword argument on HTTPSConnectionPool since version 1.25. It's definitively less ergonomic than requests, but gets the job done.

[sethmlarson]

In an effort to clean up the issue tracker to only have issues that are still relevant to the project we've done a quick pass and decided this issue may no longer be relevant for a variety of potential reasons, including:

• Applies to a much older version, unclear whether the issue still applies.
• Change requires a backwards incompatible release and it's unclear if the benefits are worth the migration effort from the community.
• There isn't a clear demand from the community on the change landing in Requests.

If you think the issue should remain open, please comment so below or open a new issue and link back to the original issue. Again, thank you for opening the issue and for the discussion, it's much appreciated.