How do I get a handler for the stored certificates

[serafdev]

So I'm writing some script that needs to connect to an external server which requires me to present a client certificate for the SSL Handshake.

My private key is stored in the HSM, I was able to use the sample code to generate a key pair but I do not see a way to add a handler to pass to my code for the SSL handshake.

Here's how my code looks like:

from pycryptoki.session_management import (
    c_initialize_ex,
    c_open_session_ex,
    login_ex,
    c_logout_ex,
    c_close_session_ex,
    c_finalize_ex,
)
from requests import Session
from zeep import Client
from zeep.transports import Transport

from app.settings import BASE_DIR

c_initialize_ex()
auth_session = c_open_session_ex(0)
login_ex(auth_session, 0, "some-pass")

session: Session = Session()
session.cert = (
    "/path/to/company1.com.pem",
    "rsa-private-156405640312",
)
session.verify = os.path.join(
    BASE_DIR, os.path.join("run", "company2.pem")
)
client: Client = Client(
    f"https://company2.com/api/v9/connector.cgi",
    transport=Transport(session=session),
)
client.settings(raw_response=True)

c_logout_ex(auth_session)
c_close_session_ex(auth_session)
c_finalize_ex()

The important line are these ones for now:

session.cert = (
    "/path/to/company1.com.pem",
    "rsa-private-156405640312",
)

Which are used to present the client's public certificate to authenticate itself, and passing as the second element of the tupple the key for decryption so the session knows how to decrypt the information.

My problem is I don't know what to put there, I tried with the label of the key in the HSM but that doesn't work, I'm guessing that I need to create a X509 object with some type of handler?

I'm not sure how to solve this and how to point to the key, any help would be great!

Thank you in advance

[astraw38]

To clarify, is session = Session() from requests? Or is it some other library (if so, which)?

Assuming it is requests, it's backended python's ssl library, and tracking that further down ref:

Load a private key and the corresponding certificate. The certfile string must be the path to a single file in PEM format containing the certificate as well as any number of CA certificates needed to establish the certificate’s authenticity. The keyfile string, if present, must point to a file containing the private key in. Otherwise the private key will be taken from certfile as well. See the discussion of Certificates for more information on how the certificate is stored in the certfile.

So I believe it should be 2 paths, 1 for the cert file, 1 for the private key. I don't think that requests provides the 'password' functionality that SSL does (see link above for details). As a matter of fact, their doc says they don't support encrypted private keys :(

[serafdev]

Thank you @astraw38 for your fast answer, it is indeed from requests, I have updated the issue.

All right I see, I thought I could create an X509 python object with a handler from pycrptoki and pass it to the Session object.

Do you have an idea if I can do this with C/C++ or just an example using the command line with curl or another library from python that supports this? I could write it with any other language too.

It would be great if you could point me to the right direction! I will keep investigating the Session object and try to extend it somehow to support encrypted certificates, thank you in advance!

[astraw38]

I believe what you'd need is an OpenSSL integration. I'm not familiar with that integration though. I would assume that most libraries that do SSL/TLS would use openssl on the backend, which would make that integration useful for all of them (and if I'm reading it right, you might be able to use the key label instead of a 'file'. I'd need to dig into python's ssl source to be sure).

It really depends a lot on what your overall goal is.

Edit: https://github.com/python/cpython/blob/40dad9545aad4ede89abbab1c1beef5303d9573e/Modules/_ssl.c#L3892 is looking like you could probably do the OpenSSL GEM integration and then use the Privkey label on the HSM as the 'private key file'. Limitations might be helpers on top of the OpenSSL calls that validate that it is a 'file'.

[astraw38]

I'll also note if you wanted to stay in python, you could likely do the wrapping of the socket on your own. It'd require some monkeypatching though.

[serafdev]

I tried what you said, I did the integration and tried the label as the "keyfile" and indeed it doesn't work, I will investigate on how I could bypass the file validator from the ssl library underneath.

Meanwhile I'm not binded to using Python, if you guys have a solution in another language (that has a compiler on linux) I will be more than glad in using it. Let me know if you have such implementation somewhere as an example!

Thank you so much @astraw38 for your time!

[mdhgarcia]

If I'm understanding correctly and what you need is to present a certificate signed by the private key in the HSM - because presumably the self-signed cert from the HSM is trusted by the server - I think I solved this particular problem (though if I don't understand correctly I've solved a different problem...). The (messy) solution looks something like the following:

1. Hand-build a TBS (to-be-signed) certificate using the pyasn1 library (this is the raw ASN.1 data that a CSR presents for signing). This involves creating a TBS header with (I believe) at least the CN, validity dates, and public key for signing.
2. Look up handles for the HSM's self-signed cert and private key by label
3. Export the HSM's self-signed cert (if you need to present the full chain)
4. Hash the TBS cert with the appropriate function
5. Sign the TBS cert hash with the HSM's private key
6. Append the signature to the TBS cert in the appropriate field to complete the full X.509 cert

If instead what you need (or maybe both) is to present a certificate to the server so it can encrypt some data for you to decrypt in the HSM, you can create a self-signed cert in the HSM using the cmu utility (I suppose you could hand-build that too and sign the HSM's own public key with its private key if you don't want to use command-line utils) and pass that to the sever.

Hope that helps! It cost me a lot of time and a hole in the wall from banging my head repeatedly...

PS: If someone at Thales would be so kind, implementing helper functions for generating a TBS cert from a public key and passing it into the HSM for signing would be much appreciated... I know cmu does this but the library doesn't AFAIK. I can also ask my employer whether they'd be willing to open-source this particular function!