Open alfonsrv opened 6 months ago
Hi @alfonsrv (and presumably others that in the future will request this),
Yes, it absolutely could be, and I'd like to support it. The major problem with it is that I simply don't have a HSM and I could not test it in anyway. So any support for HSM at present could not be tested in any meaningful scenario. If someone is willing to collaborate - sure. If someone provides me with a HSM, even better. But before that, I'm reluctant to implement anything in that direction. I don't want to promise a feature that in reality is not tested at all.
kr, Mat
I don't think having a HSM is necessary, since implementation would likely be done over another docker service e.g. Yubico's YubiHSM Python library that provides a web server to send signing requests to.
So providing a generic interface would be possible – that could either send the data to the YubiHSM (or any other HSM) – while also enabling bootstrapping extra functionality to the signing-process e.g. adding additional extensions, checking if the subject is allowed to be issued, etc.
I started working literally this week to add HSM support django-ca. I need to get a draft PR ready where we can discuss various parts about how does it work.
I am first cleaning up https://github.com/SUNET/python_x509_pkcs11 to be a bit maintainable code base and then will use the same for HSM support here.
@alfonsrv that library gives no documentation on how to actually create a signed certificate - let alone with a CSR object from python cryptography.
that takes all the required arguments
... and what would that be? From the library, I simply cannot tell. :-(. If I get the information, I might think of at least abstracting key handling away.
kr, Mat
Hi @alfonsrv and @kushaldas,
In the past days, I played around with and read the code of python-yubihsm, python-pkcs11 and python_x509_pkcs11 library. A few observations and conclusions:
python-yubihsm
lacks documentation. For example, it shows how to generate a key, but I could not figure out how to use it in a later session.python-yubihsm
requires a physical YubiHSM. Even running the test suite requires it!python-pkcs11
is in need of a new maintainer, see pyauth/python-pkcs11#169 - and it's unclear if it supports Python 3.11 and 3.12 at this point.python-pkcs11
has frustratingly incomplete/untested documentation. For example, most (all?) examples in the README simply do not work, as generating a key pair requires an R/W session and that parameter is not passed.python_x509_pkcs11
has a lot figured out of what we would need for proper supporting the PKCS11 interface. BUT it is an async
library and django-ca is not, so it could not be used out of the box.From that, I would draw the following conclusions:
Bottom line: Supporting different key storage interfaces is definitely possible, but would require a bit of refactoring. If implemented, it would provide a generic interface similar to how Django supports different databases or caches, so it would allow (in theory) @alfonsrv to implement YubiHSM support in a separate project, while PKCS11 support is included in django-ca.
Have a good weekend everybody!
kr, Mat
A generic interface could look like this:
django_ca.profiles:361
– before return builder.sign(private_key=ca.key(password), algorithm=algorithm)
is executed, the presence of a PRE_SIGN_HOOK
could be checked (function = get_hook("DJANGO_CA_PRE_SIGN_HOOK")
) and executed if valid+configuredbuilder
, and optional algorithm
as kwargsx509.Certificate
the function returns, else the regular aforementioned logic (return builder...
) is executed insteadIssue with this approach being that it's not JSON-serializable, thus cannot easily be passed to another service, such as an external HSM / some other HTTP-reachable service.
Similarly hooks for OCSP signing + CRL creation – now that I type it out, it seems like quite some overhead, but allows for only having some CA keys delegated to the HSM.
Hi @alfonsrv ,
First, a general update:
@kushaldas made significant progress on signing certificates via the PKCS11 interface with cryptography (with more help from the cryptography maintainers - thanks!). He has a proof-of-concept branch demonstrating it (@kushaldas , maybe you can link it?).
I myself worked a lot on building on Kushal's work and generalizing it. See the linked branch! My current approach is as follows:
About your idea: it's generally a valid idea and my approach isn't much different in the end. Let me explain.
The problem is that the private key is used in a variety of places. For example:
In addition, parameters differ based on implementations (password and path for filesystem, key slot and label for HSMs, ...), so we need command line integration. Now that I think of it, also in the API and admin interface.
It would mean there's lots of hooks you'd have to implement and configure for a fluent user experience.
In the end, a backend works just the same. You still have to implement everything (but with the advantage of subclass checks), but you only have to configure the path in a setting.
I'll invest more time in my branch this week. I hope to get it in good condition by Sunday. The basic concept is proven to work, but tests are not adapted. In the meantime, you're of course also welcome to fork and work on your idea, if you want.
Kr, Mat
I just merged the first version. This is pretty sophisticated and well tested already and should allow you to implement this with a subclass and a few pydantic models. Documentation is here:
https://django-ca.readthedocs.io/en/latest/python/key_backends.html
Great! I'll try to have a look into implementing a YubiHSM prototype backend over easter.
Would be super cool. I also plan to release that weekend, by the way. If there's something that needs to be changed in the key backend interface, I'm open to it.
Looking at it, there seems to be quite a lot of moving parts – cryptography doesn't support it by default, but requires the OpenSSL used underneath to utilize a PKCS11 engine (https://github.com/pyca/cryptography/issues/4967). YubiHSM seems to use libp11
(https://docs.yubico.com/hardware/yubihsm-2/hsm-2-user-guide/hsm2-openssl-libp11.html) library to provide the OpenSSL engine. (might be wrong here)
While this seems managable, I'm uncertain if implementing it as a straight-forward YubiHSM
-backend yields too many security benefits (other than preventing key extraction – which is obv pretty huge by itself), as all signing logic and HSM authentication passwords would still have to reside in the publicly facing Docker container as part of the storage configuration, allowing attackers to circumvent any kind of auditing and restrictions implemented on an application / Python-level, making signing go just as unnoticed as in the current situation.
A dedicated, air-gapped container would probably be desirable that processes "commands" sent to it – like "sign certificate with pk 2
", "create a CRL", … and so on. Especially since data has to be marshalled anyways in order to be sign-able, should the PKCS11 engine approach not work (see https://github.com/reaperhulk/vault-signing/blob/main/src/vaultsigning/key.py#L59-L66 + https://github.com/wbond/asn1crypto/issues/6)
Also will have to see if there's a way to only make cryptography
use the engine for certain operations (e.g. django-ca
) instead of all of them.
Hi @alfonsrv ,
A dedicated, air-gapped container would probably be desirable that processes "commands" sent to it – like "sign certificate with pk 2", "create a CRL", … and so on. Especially since data has to be marshalled anyways in order to be sign-able, should the PKCS11 engine approach not work (see https://github.com/reaperhulk/vault-signing/blob/main/src/vaultsigning/key.py#L59-L66 + https://github.com/wbond/asn1crypto/issues/6)
Well... I have good news for you - django-ca already has this. It supports Celery to do just that. In this mode of operation, the webserver process has no access to the private key, instead commands are sent via the broker (Redis in the examples, but could be any MQTT broker as well) to a Celery worker. The tutorials for source installation and the Docker Compose setup already include Celery.
Both processes can have different configurations, e.g. for example passwords could only be present on the Celery container.
Note however that as long as you have ACME (or the API) at the front, or want to automatically sign CRLs, you will need all configuration to sign something with the CA somewhere.
@alfonsrv , since I'll start working on @kushaldas branch, wondering if you could provide some input: What are the parameters available when generating a private key? And which would be required for using a private key for signing?
Sorry for the late reply, the email must have slipped my attention.
Creating a key and signing is quite a multi-layered process. Generally the Yubico YubiHSM documentation is quite good, but here's what's required.
Prior to usage, users first have to setup their YubiHSM with one or more authentication key, that limits the scope of operations (signing, generating + exporting keys, deleting keys, reviewing audit logs, ...) to a specific domain (effectively a cluster of private keys):
yubihsm> put authkey 0 0 "MASTER" all all all
)yubihsm> put authkey 0 102 "AuthKey (DOMAIN 3)" 3 generate-asymmetric-key:sign-pkcs:sign-pss sign-pkcs:sign-pss
)├── Root CA (auth key 1)
│ ├── Intermediate CA Identities (auth key 2)
│ │ └── Intermediate CA Identity 1 (auth key 4)
│ │ └── Intermediate CA Identity 2 (auth key 4)
│ │ └── Intermediate CA Identity 3 (auth key 4)
│ ├── Intermediate CA Web (auth key 3)
│ │ └── Intermediate CA Web-A 3 (auth key 5)
│ │ └── Intermediate CA Web-B 3 (auth key 5)
│ │ └── Intermediate CA Web-C 3 (auth key 5)
All operations require to be run in sessions, which in turn require to specify an authentication key. The authentication key basically scopes each session. Using the authentication key requires the password of that authentication key for usage. (using the CLI: yubihsm> session open <authkey_id>
)
Afterwards keys can be generated using the MASTER authentication key, or other authentication keys that were created for creating asymmetric key pairs (yubihsm> generate asymmetric 1 100 "Intermediate CA Identity 3" 3 exportable-under-wrap,sign-pkcs rsa4096
) ref.
Alternatively already-existing keys can be imported using the CLI (yubihsm> put asymmetric 1 100 "Intermediate CA Identity 3" 3 sign-pkcs private.key
)
Finally, signing works by specifying the desired asymmetric key ID + what should be signed
sign pkcs1v1_5 1 100 rsa-pkcs1-sha256 request.csr
ref
From my understanding, an OpenSSL integration should be available that makes signing of the CertificateSigningRequestBuilder
objects easier, since serializing/exporting them to a file is not possible afaik (I think it's this one https://github.com/pyca/cryptography/issues/4967 or another of the issues mentioned above).
For all of the CLI commands, the official Python wrapper provides the same functionality + syntax.
Given the complexity of initial setup and management outlined above, I think it's best if people setup the HSM via CLI and then just use Django CA for signing. This also avoids possible DoS attacks by overwriting already-existing private keys that have not been exported / backed up. The only keys that I think should be generated on-device are the OCSP keys.
I hope I could outline the process clearly enough and in an understandable way.
Hi @alfonsrv ,
Does YubiHSM support that protocol as well?
kr, Mat
Could it be a feature request to sign certificates not via private keys saved on the file system, but by delegating signing to another function that can be specified in
settings.py
(e.g.external.services.sign
) that takes all the required arguments and returns the signed certificate?Main reason being so signing can happen via a HSM (e.g. YubiHSM) instead of having to rely on locally saved private key files – which even if they are encrypted just don't feel as save as delegating it to a dedicated HSM.