bug: SSL_CTX_use_certificate throws key values mismatch only for EC keys

[bshaffer]

Hello! Thank you for taking the time to read this!

Overview

We are trying to use OpenSSL._util.lib.SSL_CTX_use_PrivateKey with an EC key, and are getting the following exception:

Exception: [('elliptic curve routines', 'EC_POINT_cmp', 'incompatible objects'), ('x509 certificate routines', 'X509_check_private_key', 'key values mismatch')]

The strange thing is:

• We can get this to work when using RSA keys instead of EC keys
• We can get this to work when we use SSL_CTX_use_PrivateKey_file instead of SSL_CTX_use_PrivateKey.

These two things have lead us to believe this is a bug and not user error.

We are the a pkcs11 module SoftHSM, but this should be repeatable using any pkcs11 module. We have tested on both Linux and Mac, but not with any other pkcs11 module.

Steps to reproduce

1. Generate an EC key and certificate:

   openssl ecparam -noout -name prime256v1 -genkey -out ec-key.pem -outform PEM
   openssl req -new -x509 -key ec-key.pem -out ec-cert.pem -days 730

2. Store the key and cert as eckey into your pkcs11 module:

   pkcs11-tool  --module /usr/local/lib/softhsm/libsofthsm2.so --pin realgoodpin \
   --write-object ec-key.pem --type privkey --id 3333 --label eckey --slot-index 0
   pkcs11-tool  --module /usr/local/lib/softhsm/libsofthsm2.so --pin realgoodpin \
   --write-object ec-cert.pem --type cert --id 3333 --label eckey --slot-index 0

3. Execute the following code. You will get a "key values mismatch" exception:

   
   import OpenSSL.SSL
   from OpenSSL._util import exception_from_error_queue as _exception_from_error_queue
   from OpenSSL._util import (
   ffi as _ffi,
   lib as _lib,
   )

cert_path="./ec-cert.pem" key_id = b"pkcs11:token=token1;object=eckey;pin-value=realgoodpin" print("============ testing using key from hsm ================")

_lib.ENGINE_load_builtin_engines() e = _lib.ENGINE_by_id(b"dynamic") _lib.ENGINE_ctrl_cmd_string(e, b"ID", b"pkcs11", 0) _lib.ENGINE_ctrl_cmd_string(e, b"SO_PATH", b"/usr/lib/x86_64-linux-gnu/engines-1.1/libpkcs11.so", 0) _lib.ENGINE_ctrl_cmd_string(e, b"LOAD", _ffi.NULL, 0) _lib.ENGINE_ctrl_cmd_string(e, b"MODULE_PATH", b"/usr/local/lib/softhsm/libsofthsm2.so", 0) _lib.ENGINE_init(e)

context = OpenSSL.SSL.Context(6)

_lib.SSL_CTX_use_certificate_file(context._context, cert_path, 1) key = _lib.ENGINE_load_private_key(e, key_id, _ffi.NULL, _ffi.NULL) if not _lib.SSL_CTX_use_PrivateKey(context._context, key): _exception_from_error_queue(Exception) print("test_use_key_from_hsm succeeded")

### Using RSA keys instead work

1. Generate RSA instead of EC:

generate RSA cert/key

openssl req -nodes -x509 -newkey rsa:4096 -keyout rsa-key.pem -out rsa-cert.pem -days 3650

2. Save them as `rsakey` to your pkcs11 module:

pkcs11-tool --module /usr/local/lib/softhsm/libsofthsm2.so --pin realgoodpin \ --write-object rsa-key.pem --type privkey --id 2222 --label rsakey --slot-index 0 pkcs11-tool --module /usr/local/lib/softhsm/libsofthsm2.so --pin realgoodpin \ --write-object rsa-cert.pem --type cert --id 2222 --label rsakey --slot-index 0

3. Run the _exact same code_ as in Step 3 above, but change `eckey` in your `key_id` variable to `rsakey`, and the `cert_path` variable to `"./rsa-cert.pem"`. The code will execute without an exception.

### Using the same EC Key / Cert as a file works

Now try the same code using an EC key, but instead of using `SSL_CTX_use_PrivateKey`, use `SSL_CTX_use_PrivateKey_file`  (and remove the call on the proceeding line to `ENGINE_load_private_key` as we no longer need it)

```python
# key = _lib.ENGINE_load_private_key(e, key_id, _ffi.NULL, _ffi.NULL)
# if not _lib.SSL_CTX_use_PrivateKey(context._context, key):
if not _lib.SSL_CTX_use_PrivateKey_file(context._context, "./ec-key.pem", 1):

If there's anything obvious we are doing wrong please let us know! Any help is appreciated! At this point it does seem like a bug. Thank you!

[reaperhulk]

The behavior you're seeing is within OpenSSL itself, not pyOpenSSL. To debug this effectively you'll need to rewrite this in C (which it mostly is already!), and if you're still seeing the problem seek help from the OpenSSL user community. If, for whatever reason, you don't see this when translating into pure C then please reopen this issue so we can dig deeper!