OpenSSL 3 support for loading engine keys

[DCrow]

Openssl 3 still supports loading engines and some engines still haven't migrated to using providers. In such cases it is possible to continue using engines. They can be loaded using openssl's config file.

Previously I could load a private key(provided by custom engine) like so

  OpenSSL::PKey.read("some private key")

But now even though engine is loaded the same command returns Could not parse PKey (OpenSSL::PKey::PKeyError).

Using OpenSSL::PKey.new_raw_private_key also did not help. Locally installing ruby/openssl and reverting ossl_pkey_read_generic in ossl_pkey.c to the previous version(before openssl 3 support) did help, but this method is not viable for production.

Is it possible to provide some way to load such keys?

[rhenium]

Related to #722.

Could you tell us which engine you are using?

[DCrow]

Hello, I am using GOST-engine.

Currently it is in the process of migrating to openssl 3 but it's still incomplete.

GOST-engine maintainer has recommended to load private keys using PEM_read_bio_PrivateKey which is used in ruby/openssl(but only for openssl < 3).

Here's an example with a private key which ruby/openssl (openssl >= 3) doesn't read but openssl says is valid.

openssl req -x509 -newkey gost2012_256 -pkeyopt paramset:A -nodes -keyout gost_private.pem -out gost_cert.crt -subj "/C=RU/ST=Test/L=Test/O=Test/OU=Test/CN=example.com"

openssl pkey -in ../gost_private.pem -check

For now I am using this to accomplish compatibility with engine provided keys, i.e. trying to read the key using the old way in addition to currently used way.

[rhenium]

I reproduced the error using the gost engine with OpenSSL 3.2.

According to https://github.com/gost-engine/engine/blob/e0a500ab877ba72cb14026a24d462dd923b90ced/README.prov.md, pkeys haven't been ported to a provider and are only implemented through the legacy engine interface.

$ openssl genpkey -engine gost -algorithm gost2012_256 -pkeyopt paramset:A >gost2012_256.pem
$ ruby -Ilib -ropenssl -e'OpenSSL.debug=true; p OpenSSL::PKey.read(File.read("./gost2012_256.pem"))'
-e:1: warning: error on stack: error:1E08010C:DECODER routines:OSSL_DECODER_from_bio:unsupported (No supported data to decode. Input type: DER)
-e:1: warning: error on stack: error:1E08010C:DECODER routines:OSSL_DECODER_from_bio:unsupported (No supported data to decode. Input type: DER)
-e:1: warning: error on stack: error:1E08010C:DECODER routines:OSSL_DECODER_from_bio:unsupported (No supported data to decode. Input type: DER)
-e:1: warning: error on stack: error:1E08010C:DECODER routines:OSSL_DECODER_from_bio:unsupported (No supported data to decode. Input type: DER)
-e:1: warning: error on stack: error:1E08010C:DECODER routines:OSSL_DECODER_from_bio:unsupported (No supported data to decode. Input type: DER)
-e:1: warning: error on stack: error:1E08010C:DECODER routines:OSSL_DECODER_from_bio:unsupported (No supported data to decode. Input type: DER)
-e:1: warning: error on stack: error:1E08010C:DECODER routines:OSSL_DECODER_from_bio:unsupported (No supported data to decode. Input type: DER)
-e:1: warning: error on stack: error:1E08010C:DECODER routines:OSSL_DECODER_from_bio:unsupported (No supported data to decode. Input type: DER)
-e:1: warning: error on stack: error:1E08010C:DECODER routines:OSSL_DECODER_from_bio:unsupported (No supported data to decode. Input type: DER)
-e:1: warning: error on stack: error:1E08010C:DECODER routines:OSSL_DECODER_from_bio:unsupported (No supported data to decode. Input type: PEM)
-e:1: warning: error on stack: error:1E08010C:DECODER routines:OSSL_DECODER_from_bio:unsupported (No supported data to decode. Input type: PEM)
-e:1: warning: error on stack: error:1E08010C:DECODER routines:OSSL_DECODER_from_bio:unsupported (No supported data to decode. Input type: PEM)
-e:1:in `read': Could not parse PKey (OpenSSL::PKey::PKeyError)
    from -e:1:in `<main>'

OpenSSL::PKey.read currently entirely relies on the OSSL_DECODER API. This API apparently doesn't take care of legacy pkeys (i.e., not backed by a provider). I've overlooked it.

https://github.com/ruby/openssl/blob/043c503750e5fcf2e3e186a458b5d6012ebdb98e/ext/openssl/ossl_pkey.c#L82-L181

It looks like we have to fallback to legacy PEM decoding with EVP_PKEY_ASN1_METHOD if OSSL_DECODER fails.