Suggested edit for YubiHSM2/index

[Jiang-BW]

Hi, there,

In https://developers.yubico.com/YubiHSM2/Usage_Guides/OpenSSL_with_libp11.html, could you please give an example commands of using openssl and yubihsm to generate the key pair and the self-signed Root CA cert and then use openssl ca to sign a CSR? Currently I couldn't find how to set the parameters of these openssl commands to use yubihsm keys:

openssl req -new -newkey rsa:4096 -x509 -config RootCA.conf -nodes -days 7300 -keyout RootCA_PriK.key -out RootCACert.pem -outform PEM -set_serial 0x1 -extensions my_extensions

openssl ca -config CASign.conf -policy signing_policy -extensions signing_req -infiles myCSR.csr

In a nutshell, the question is how to let "openssl ca" to use the pkcs11 engine to sign the certificate?

[a-dma]

key generation in OpenSSL is broken (-keygen_engine in req doesn't really behave as it should). Common practice is to use something else to generate the key and then tell OpenSSL to use pkcs11 engine to access the key.

Generate an RSA key using either pkcs11-tool or yubihsm-shell and give it the label testkey. After that you can run the command openssl req -new -x509 -config engine.conf -nodes -days 7300 -out RootCACert.pem -outform PEM -set_serial 0x1 -engine pkcs11 -keyform engine -key label_testkey

where engine.conf is something like this

openssl_conf = openssl_init
[openssl_init]
engines = engine_section

[engine_section]
pkcs11 = pkcs11_section

[pkcs11_section]
engine_id = pkcs11
dynamic_path = /usr/lib/x86_64-linux-gnu/openssl-1.0.2/engines/libpkcs11.so
MODULE_PATH = /somewhere/yubihsm_pkcs11.so # don't forget the config file for the PKCS#11 module
PIN = 0001password
init = 0

[req]
distinguished_name = req_distinguished_name
prompt = no

[req_distinguished_name]
countryName             = AA
stateOrProvinceName     = foo
localityName            = bar

a similar command should also work for openssl ca.

[Jiang-BW]

Hi, there,

It seems not working:

[jzhang@bwcalaptop-localdomain BWCA_Device]$ openssl req -new -x509 -config ./Config/engine.conf -nodes -days 7300 -out Certs/cert.pem -set_serial 0x01 -engine pkcs11 -keyform engine -key Device_RootCA1_Key engine "pkcs11" set. Format not recognized! The certificate ID is not a valid PKCS#11 URI The PKCS#11 URI format is defined by RFC7512 The legacy ENGINE_pkcs11 ID format is also still accepted for now Format not recognized! The certificate ID is not a valid PKCS#11 URI The PKCS#11 URI format is defined by RFC7512 The legacy ENGINE_pkcs11 ID format is also still accepted for now PKCS11_get_private_key returned NULL cannot load Private Key from engine 140601890260896:error:80065064:pkcs11 engine:ctx_load_key:invalid id:eng_back.c:651: 140601890260896:error:26096080:engine routines:ENGINE_load_private_key:failed loading private key:eng_pkey.c:124: unable to load Private Key

[jzhang@bwcalaptop-localdomain BWCA_Device]$ openssl ca -config ./Config/Device_SubCA_Sign.conf -policy signing_policy -extensions signing_req -infiles Certs/BWSubRootCA_Cert_Device.csr Using configuration from ./Config/Device_SubCA_Sign.conf Error opening CA private key pkcs11 139620818667424:error:02001002:system library:fopen:No such file or directory:bss_file.c:402:fopen('pkcs11','r') 139620818667424:error:20074002:BIO routines:FILE_CTRL:system lib:bss_file.c:404: unable to load CA private key [jzhang@bwcalaptop-localdomain BWCA_Device]$

The ./Config/engine.conf is created following instruction and the ./Config/Device_SubCA_Sign.conf is attached.

Best Regards,

Jiang

On Mar 2, 2018, at 2:42 AM, Alessio Di Mauro notifications@github.com wrote:

key generation in OpenSSL is broken (-keygen_engine in req doesn't really behave as it should). Common practice is to use something else to generate the key and then tell OpenSSL to use pkcs11 engine to access the key.

Generate an RSA key using either pkcs11-tool or yubihsm-shell and give it the label testkey. After that you can run the command openssl req -new -x509 -config engine.conf -nodes -days 7300 -out RootCACert.pem -outform PEM -set_serial 0x1 -engine pkcs11 -keyform engine -key label_testkey

where engine.conf is something like this

openssl_conf = openssl_init [openssl_init] engines = engine_section

[engine_section] pkcs11 = pkcs11_section

[pkcs11_section] engine_id = pkcs11 dynamic_path = /usr/lib/x86_64-linux-gnu/openssl-1.0.2/engines/libpkcs11.so MODULE_PATH = /somewhere/yubihsm_pkcs11.so # don't forget the config file for the PKCS#11 module PIN = 0001password init = 0

[req] distinguished_name = req_distinguished_name prompt = no

[req_distinguished_name] countryName = AA stateOrProvinceName = foo localityName = bar a similar command should also work for openssl ca.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Yubico/developers.yubico.com/issues/109#issuecomment-369887461, or mute the thread https://github.com/notifications/unsubscribe-auth/Ag3HcjUpaYppS9bUhiFvJgtfIuewG78iks5taSIMgaJpZM4SXpjH.

[a-dma]

I don't think GitHub issues support attachments. Please paste both configuration files in the body of the issue. Also what versions of OpenSSL and PKCS#11 Engine are you using?

The fact that the engine is set but the format is not recognized makes me think it's not compatible.

For reference, I'm using OpenSSL 1.0.2g and PKCS#11 Engine 0.4.7

[Jiang-BW]

Hi, Alessio,

Here’s the other config file:

I am using CentOS 7 with openssl 1.0.2k-fips and PKCS11 0.4.7.

Best Regards,

Jiang

On Mar 6, 2018, at 6:08 AM, Alessio Di Mauro notifications@github.com wrote:

I don't think GitHub issues support attachments. Please paste both configuration files in the body of the issue. Also what versions of OpenSSL and PKCS#11 Engine are you using?

The fact that the engine is set but the format is not recognized makes me think it's not compatible.

For reference, I'm using OpenSSL 1.0.2g and PKCS#11 Engine 0.4.7

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Yubico/developers.yubico.com/issues/109#issuecomment-370792394, or mute the thread https://github.com/notifications/unsubscribe-auth/Ag3HcpO-AP-zqAUE5SaelRAklaUe1YLMks5tbphtgaJpZM4SXpjH.

[a-dma]

Sorry but the config file is still missing. Try to reply from GitHub's website rather than from your email client. Link to the issue is https://github.com/Yubico/developers.yubico.com/issues/109

[Jiang-BW]

Here's the engine.conf:

################################### openssl_conf = openssl_init [openssl_init] engines = engine_section

[engine_section] pkcs11 = pkcs11_section

[pkcs11_section] engine_id = pkcs11 dynamic_path = /usr/lib64/openssl/engines/libpkcs11.so MODULE_PATH = /home/jzhang/HSM/yubihsm2-sdk/lib/yubihsm_pkcs11.so PIN = 0001password init = 0

[req] distinguished_name = req_distinguished_name prompt = no

[req_distinguished_name] countryName = AA stateOrProvinceName = foo localityName = bar

#######################################

Here's the Device_SubCA_Sign.conf:

######################## openssl_conf = openssl_init [openssl_init] engines = engine_section

[engine_section] pkcs11 = pkcs11_section

[pkcs11_section] engine_id = pkcs11 dynamic_path = /usr/lib64/openssl/engines/libpkcs11.so MODULE_PATH = /home/jzhang/HSM/yubihsm2-sdk/lib/yubihsm_pkcs11.so PIN = 0001password init = 0

[ ca ] default_ca = my_ca

[ my_ca ] serial = ./Data/serial_SubCA

database = ./Data/index.txt

new_certs_dir = ./Certs/

certificate = ./Certs/DeviceRootCACert.pem

private_key = pkcs11

default_md = sha256

default_days = 7300

policy = signing_policy

prompt = no

[ signing_policy ] countryName = match stateOrProvinceName = supplied organizationName = supplied commonName = supplied organizationalUnitName = optional commonName = supplied

[ signing_req ] subjectKeyIdentifier = hash authorityKeyIdentifier = keyid,issuer basicConstraints = critical,@bs_section keyUsage = critical, keyCertSign

[bs_section] CA = TRUE pathlen = 0

[ my_subject_alt_names ] DNS = Engineering Device Sub CA 1 ########################

[Jiang-BW]

OK. Posted on the link

Best Regards,

Jiang

On Mar 7, 2018, at 3:41 AM, Alessio Di Mauro notifications@github.com wrote:

Sorry but the config file is still missing. Try to reply from GitHub's website rather than from your email client. Link to the issue is #109 https://github.com/Yubico/developers.yubico.com/issues/109 — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Yubico/developers.yubico.com/issues/109#issuecomment-371112337, or mute the thread https://github.com/notifications/unsubscribe-auth/Ag3Hcsk3Q11K4hgytrPr4Crb4Nq6CDH5ks5tb8dPgaJpZM4SXpjH.

[a-dma]

Thanks, that's better.

In your command

openssl req -new -x509 -config ./Config/engine.conf -nodes -days 7300 -out Certs/cert.pem -set_serial 0x01 -engine pkcs11 -keyform engine -key Device_RootCA1_Key

you specify the key to be Device_RootCA1_Key. If you use that syntax, you're trying to access a file. To use the key from the HSM you need to use the syntax label_xxx or id_abcd. If you scroll up in one of my previous comments you can see that I mentioned creating a key with label testkey and then I use it in a command by doing -key label_testkey. Note that this is how PKCS#11 Engine works, it's not a YubiHSM-specific behavior.

The error in the second command is similar, in the configuration file you're specifying the key as private_key = pkcs11, which means OpenSSL is looking for a file called pkcs11.

[r4yv3n]

I'm having the same issue as well but, only when using 'open ca' to create the intermediate CA. I have no problem with any other openssl operation. For those successful functions I can do both label_ and 0:0001. I've only been playing around with this for a couple of days so I could definitely be missing something. Hopefully someone with more experience will spot the issue.

I've tried the following default Ubuntu packages as well as latest from source: openssl-1.0.2n openssl-1.1.1-pre2 libp11-0.3.1-1 (pkg) libp11-0.4.0

openssl ca -config ./openssl.cnf -extensions v3_intermediate_ca -days 1825 -notext -md sha256 -key 0:0001 -keyform engine -engine pkcs11 -in intermediate/csr/first.intermediate.csr.pem -out test.pem

Regardless whether private_key is pointing to a file or pkcs11 it works the same with the exception of "open ca"

Using configuration from ./openssl.cnf engine "pkcs11" set. Format not recognized! The key ID is not a valid PKCS#11 URI The PKCS#11 URI format is defined by RFC7512 The legacy ENGINE_pkcs11 ID format is also still accepted for now Format not recognized! The key ID is not a valid PKCS#11 URI The PKCS#11 URI format is defined by RFC7512 The legacy ENGINE_pkcs11 ID format is also still accepted for now PKCS11_get_private_key returned NULL cannot load CA private key from engine 140307159651992:error:80065064:pkcs11 engine:ctx_load_key:invalid id:eng_back.c:660: 140307159651992:error:26096080:engine routines:ENGINE_load_private_key:failed loading private key:eng_pkey.c:124:

----------------- engine.conf --------------- openssl_conf = openssl_init

[openssl_init] engines = engine_section

[engine_section] pkcs11 = pkcs11_section

[pkcs11_section] engine_id = pkcs11 dynamic_path = /usr/lib/engines/engine_pkcs11.so

dynamic_path = /usr/lib/engines/pkcs11.so

MODULE_PATH = /usr/lib/x86_64-linux-gnu/pkcs11/yubihsm_pkcs11.so

---

OPENSSL_CONF=openssl.cnf openssl engine -t -c pkcs11

(pkcs11) pkcs11 engine [RSA, rsaEncryption] [ available ] --------------------

pkcs11-tool --module /usr/lib/x86_64-linux-gnu/pkcs11/yubihsm_pkcs11.so -l --pin 0001xxxxxxxxxxxxxxxxxx -O -T -L Available slots: Slot 0 (0x0): YubiHSM Connector 127.0.0.1 token label : YubiHSM token manufacturer : Yubico (www.yubico.com) token model : YubiHSM token flags : rng, login required, PIN initialized, token initialized hardware version : 2.0 firmware version : 2.0 serial num : 07040338 Using slot 0 with a present token (0x0) Private Key Object; RSA label: RSA Root CA Key ID: 0001 Usage: sign Public Key Object; RSA 4096 bits label: RSA Root CA Key ID: 0001 Usage: verify

--------------- openssl.cnf --------------------------
[openssl_init] engines = engine_section [engine_section] pkcs11 = pkcs11_section [pkcs11_section] engine_id = pkcs11 dynamic_path = /usr/lib/engines/pkcs11.so MODULE_PATH = /usr/lib/x86_64-linux-gnu/pkcs11/yubihsm_pkcs11.so init = 0 [ ca ] default_ca = CA_default [ CA_default ] baseroot = /opt dir = $baseroot/root/ca certs = $dir/certs crl_dir = $dir/crl new_certs_dir = $dir/newcerts database = $dir/index.txt serial = $dir/serial RANDFILE = $dir/private/.rand private_key = pkcs11 certificate = $dir/certs/ca.pem crlnumber = $dir/crlnumber crl = $dir/crl/ca.crl.pem crl_extensions = crl_ext default_crl_days = 30 default_md = sha256 name_opt = ca_default cert_opt = ca_default default_days = 375 preserve = no policy = policy_strict [ policy_strict ] countryName = match stateOrProvinceName = match organizationName = match organizationalUnitName = optional commonName = supplied emailAddress = optional [ policy_loose ] countryName = optional stateOrProvinceName = optional localityName = optional organizationName = optional organizationalUnitName = optional commonName = supplied emailAddress = optional [ req ] default_bits = 2048 distinguished_name = req_distinguished_name string_mask = utf8only default_md = sha256 x509_extensions = v3_ca [ req_distinguished_name ] countryName = Country Name (2 letter code) stateOrProvinceName = State or Province Name localityName = Locality Name 0.organizationName = Organization Name organizationalUnitName = Organizational Unit Name commonName = Common Name emailAddress = Email Address countryName_default = US stateOrProvinceName_default = CA localityName_default = xxxxxxxxxxx 0.organizationName_default = xxxxxxxxxx emailAddress_default = xxxxxxxxxxxxx [ v3_ca ] subjectKeyIdentifier = hash authorityKeyIdentifier = keyid:always,issuer basicConstraints = critical, CA:true keyUsage = critical, digitalSignature, cRLSign, keyCertSign [ v3_intermediate_ca ] subjectKeyIdentifier = hash authorityKeyIdentifier = keyid:always,issuer basicConstraints = critical, CA:true, pathlen:0 keyUsage = critical, digitalSignature, cRLSign, keyCertSign [ usr_cert ] basicConstraints = CA:FALSE nsCertType = client, email nsComment = "OpenSSL Generated Client Certificate" subjectKeyIdentifier = hash authorityKeyIdentifier = keyid,issuer keyUsage = critical, nonRepudiation, digitalSignature, keyEncipherment extendedKeyUsage = clientAuth, emailProtection [ server_cert ] basicConstraints = CA:FALSE nsCertType = server nsComment = "OpenSSL Generated Server Certificate" subjectKeyIdentifier = hash authorityKeyIdentifier = keyid,issuer:always keyUsage = critical, digitalSignature, keyEncipherment extendedKeyUsage = serverAuth [ crl_ext ] authorityKeyIdentifier=keyid:always [ ocsp ] basicConstraints = CA:FALSE subjectKeyIdentifier = hash authorityKeyIdentifier = keyid,issuer keyUsage = critical, digitalSignature extendedKeyUsage = critical, OCSPSigning

[a-dma]

The formatting is a bit off, I think you had comments in there that got parsed into MarkDown. Please use proper formatting and check the preview when adding long blocks.

As far as I understand you have problems with this command? openssl ca -config ./openssl.cnf -extensions v3_intermediate_ca -days 1825 -notext -md sha256 -key 0:0001 -keyform engine -engine pkcs11 -in intermediate/csr/first.intermediate.csr.pem -out test.pem

What do you mean with open ca? Is that just openssl ca?

Libp11 can use a different URI-based format for resource definition. Something like pkcs11:token=YubiHSM;id=%04%01;type=private. I took that example for the syntax from here and more can be found in RFC7512 as mentioned in the error message. See if that helps. It could also be a limitation in OpenSSL or a problem with our PKCS#11 module. If you think that's the case, try enabling debug for the module and see if that produces any error.

[r4yv3n]

Hi Alessio,

Yeah I haven't posted on github before. That was literally my first so I'll go back and fix it.

Here is a list of commands that work just fine for me with "-key 0:0001 -keyform engine -engine pkcs11"

openssl genrsa openssl req

If only fails using "openssl ca"

By and large those commands above are almost all you'll ever need if all you do is create a self-sign CA and then generate certs directly. However, if you want to create intermediate CA's you need "openssl ca". This is where it fails. I've put yubihsm-connect in debug mode and what I see is nothing being send to the connect from libp11 or the yubihsm module accept a ping of some sort that is this data only in hex " V ‚bb !"#$%&'()0123456789@ABCDEFG".

The return response must then be negative but it is demonstrated by openssl as a failure to provide key or keyform information. I think this is just a default.

I'll try to sort it out as far as libp11 but it would really be helpful to have access to the yubihsm pkcs11 module to rule out any issues there.

On Wed, Mar 14, 2018 at 3:23 AM, Alessio Di Mauro notifications@github.com wrote:

The formatting is a bit off, I think you had comments in there that got parsed into MarkDown. Please use proper formatting https://github.github.com/gfm/#code-fence and check the preview when adding long blocks.

As far as I understand you have problems with this command? openssl ca -config ./openssl.cnf -extensions v3_intermediate_ca -days 1825 -notext -md sha256 -key 0:0001 -keyform engine -engine pkcs11 -in intermediate/csr/first.intermediate.csr.pem -out test.pem

What do you mean with open ca? Is that just openssl ca?

Libp11 can use a different URI-based format for resource definition. Something like pkcs11:token=YubiHSM;id=%04%01;type=private. I took that example for the syntax from here https://developers.yubico.com/YubiHSM2/Usage_Guides/OpenSSL_with_libp11.html and more can be found in RFC7512 https://tools.ietf.org/html/rfc7512 as mentioned in the error message. See if that helps. It could also be a limitation in OpenSSL or a problem with our PKCS#11 module. If you think that's the case, try enabling debug for the module and see if that produces any error.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Yubico/developers.yubico.com/issues/109#issuecomment-372972646, or mute the thread https://github.com/notifications/unsubscribe-auth/AGg2u2dGU-HPhowCn6Mm0M-05RdC7fJlks5teO-ygaJpZM4SXpjH .

-- You are FREE to become a slave

Key ID: 9A452ABAA4593489 Finger Print: 7A8A 5849 ED44 52B1 0D8A EDAC 9A45 2ABA A459 3489 Pub Key: http://pgp.mit.edu:11371/pks/lookup?search=rayv33n%40gmail.com&op=index

[r4yv3n]

Figured it out! I was pretty fixated on openssl commands using "-key" but for 'openssl ca' at least you'll use -keyfile. Not for forgetting to choose the Root CA's asymmetric id.

BTW I was able to get this working using all three of the methods available.

-sign pkcs11:token=YubiHSM;id=0:0001;type=private or -key 0:001 -engine pkcs11 -keyform engine depending on command it maybe -keyfile or -key label_'My Root CA Label'

Better documentation would probably go a long way to helping first time noobs like me. But, all things considered YubiHSM is pretty cool!

Thanks for your help!

[a-dma]

Glad you got it working!

OpenSSL gets very complicated very quickly when you start dealing with CA stuff, and PKCS#11 doesn't make things better usually...

If you're willing to put together a series of steps in a coherent example/mini tutorial, feel free to send it our way: we'd be happy to put it up on the website.

[r4yv3n]

I already have the notes put together for an internal document and would be happy to share but I would especially appreciate getting some answers to questions I send Chris and Finney because I want to add those answers to this document for anyone beginning to user the YubiHSM in this way.

I would also be willing to share my discoveries concern how I will make this all work with the new AWS ACM. thanks again!

[Jiang-BW]

Hi, Alessio,

You mentioned "The error in the second command is similar, in the configuration file you're specifying the key as private_key = pkcs11, which means OpenSSL is looking for a file called pkcs11." So the question is: what should we set to the "private_key" if using pkcs11 engine. It seems this item is mandatory. I cannot comment it out.

[Jiang-BW]

BTW, the private_key option is not needed in "openssl req" but it is needed in "openssl ca".

[Jiang-BW]

On Mar 8, 2018, at 12:54 AM, Alessio Di Mauro notifications@github.com wrote:

Thanks, that's better.

In your command

openssl req -new -x509 -config ./Config/engine.conf -nodes -days 7300 -out Certs/cert.pem -set_serial 0x01 -engine pkcs11 -keyform engine -key Device_RootCA1_Key you specify the key to be Device_RootCA1_Key. If you use that syntax, you're trying to access a file. To use the key from the HSM you need to use the syntax label_xxx or id_abcd. If you scroll up in one of my previous comments you can see that I mentioned creating a key with label testkey and then I use it in a command by doing -key label_testkey. Note that this is how PKCS#11 Engine works, it's not a YubiHSM-specific behavior.

The error in the second command is similar, in the configuration file you're specifying the key as private_key = pkcs11, which means OpenSSL is looking for a file called pkcs11.

What should I set to the “private_key” if using the pkcs11 engine. It seems I cannot comment it out but it always consider as a file name. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Yubico/developers.yubico.com/issues/109#issuecomment-371421985, or mute the thread https://github.com/notifications/unsubscribe-auth/Ag3HciWBC6F44shcjAI280HEUQ0bVDLEks5tcPGwgaJpZM4SXpjH.

[r4yv3n]

I've set my private_key to pkcs11 and notarealfile.pem and it didn't matter using the openssl ca command I used it seems not to care.

[Jiang-BW]

How do you know you are using the Private Key in HSM? I think the openssl ca opens the private key key and verify if it's the matching key for the RootCA certificate's public key you passed in.

[r4yv3n]

openssl ca -config openssl.cnf -keyfile SLOT_KeyIdOfRootCA -extensions v3_intermediate_ca -engine pkcs11 -keyform engine -days 1825 -notext -md sha256 -in intermediate/csr/test_intermediate.csr.pem -out intermediate/certs/test_intermediate.cert

if SLOT_KeyIdOfRootCA is anything but the slot and key id of my private it. It fails.

[blaufish]

Edit: My original post was user error. as mentioned above by r4yv3n, openssl ca -keyfile should be used, not openssl ca -key. -key has inconsistent meaning between different openssl commands, and for ca it means password, not key.

---

openssl ca appears to require PIN = 0001password (or whatever your PIN is).

Otherwise you get errors similar to this:

Login failed
Login to token failed, returning NULL...
PKCS11_get_private_key returned NULL
cannot load CA private key from engine
140735853761408:error:28078064:UI routines:UI_set_result_ex:result too large:crypto/ui/ui_lib.c:910:You must type in 4 to 32 characters
140735853761408:error:82074007:PKCS#11 module:pkcs11_login:Invalid arguments:p11_slot.c:240: 
140735853761408:error:26096080:engine routines:ENGINE_load_private_key:failed loading private key:crypto/engine/eng_pkey.c:78:
unable to load CA private key

Anyone aware of a method of getting openssl ca not requiring PIN in configuration file?