search

latchset / pkcs11-provider

A pkcs#11 provider for OpenSSL 3.0+
Other
65 stars 39 forks source link

Fails to encrypt using pubkey on YubiHSM #198

Closed mouse07410 closed 1 year ago

mouse07410 commented 1 year ago

Describe the bug

Fails to encrypt (RSA OAEP) using public key on HSM device:

$ openssl pkeyutl -encrypt -pubin -inkey "pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;object=RSA-OAEP;type=public" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -in /tmp/key.bin -out /tmp/key.enc1
Enter pass phrase for PKCS#11 Token (Slot 0 - YubiHSM Connector 0.0.0.0):
Public Key operation error
8096FA56F87F0000:error:42000063:pkcs11:p11prov_EncryptInit:The specified key is not the correct type of key to use with the specified mechanism:./interface.gen.c:413:Error returned by C_EncryptInit

To Reproduce

$ openssl rand -out /tmp/key.bin 32
$ openssl pkeyutl -encrypt -pubin -inkey "pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;object=RSA-OAEP;type=public" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -in /tmp/key.bin -out /tmp/key.enc1

Observe the result.

Expected behavior

Sucessful encryption

$ openssl pkeyutl -engine pkcs11 -keyform engine -encrypt -pubin -inkey "pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;object=RSA-OAEP;type=public" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -in /tmp/key.bin -out /tmp/key.enc1
Engine "pkcs11" set.
Enter PKCS#11 token PIN for YubiHSM:
$ ll /tmp/key.enc1
-rw-r--r--  1 ur20980  wheel  384 Feb 15 12:04 /tmp/key.enc1
$

Operating environment (please complete the following information):

Token and application used (please complete the following information):

Additional context

None.

simo5 commented 1 year ago

Just so that it doesn't look this is being ignored. In another bug we made a first pass analysis and can't find anything obviously wrong about the pkcs11 code. There is a chance this is a driver bug. But until I can reproduce, (or someone that can figures out what is wrong) we are a bit stuck here.

mouse07410 commented 1 year ago

Just so that it doesn't look this is being ignored.

Thank you!

can't find anything obviously wrong about the pkcs11 code

Did you try adding prompt for the PIN and re-trying the operation (on public object) if access failed...?

There is a chance this is a driver bug.

My reason to doubt this is that the libp11 engine seems to work correctly with this same driver.

But reproducing it on your hardware without having an HSM is no fun.

Would your company consider purchasing a YubiHSM2 device (ballpark $600, if memory serves)? It's a nice and useful device, they probably won't regret buying it...

simo5 commented 1 year ago

Did you try adding prompt for the PIN and re-trying the operation (on public object) if access failed...?

If a public key is not found we retry with login already. See #197

Retrying the whole operation is not an option, the operation will not even happen if a public key is not available.

There is a chance this is a driver bug.

My reason to doubt this is that the libp11 engine seems to work correctly with this same driver.

libp11 is not actually doing this operation as you can see from the traces, OpenSSL exports the public key to the default provider and then performs the operation on its own.

But reproducing it on your hardware is no fun.

Would your company consider purchasing a YubiHSM2 device (ballpark $600, if memory serves)? It's a nice and useful device, they probably won't regret buying it...

I am already trying to source one, and may be able to test with a different HSM, it will just take some time.

mouse07410 commented 1 year ago

OpenSSL exports the public key to the default provider and then performs the operation on its own.

Would it make sense to have pkcs11-provider do the same? Since, AFAIK, no actual token (probably, including HSM???) would perform "public" operation on-board? Aka, "decrypt" and "sign" - on-board, but "encrypt" and "verify" - passed along to the software?

dengert commented 1 year ago

If anyone has not see these, here is some interesting YubiHSM2 info:

simo5 commented 1 year ago

Sounds to me yubiHSM2 is a non-functional pkcs11 device (unless they fix these shortcomings in their pkcs11 driver).

"When using SunPKCS11 provider, it’s important to know that generating asymmetric keys using C_GenerateKeyPair will not work."

This thing is a joke, sorry, not much point in wasting a lot of time to make it work for now...

dengert commented 1 year ago

The source code is at https://github.com/Yubico/yubihsm-shell/tree/master/pkcs11

And links against 1.1.1 on linux: https://github.com/Yubico/yubihsm-shell/blob/master/CMakeLists.txt#L163

simo5 commented 1 year ago

I am not well versed in cmake, do they statically build libcrypto into the driver?

Or do they expect a compatible OpenSSL version to always be available? (good luck with that)

dengert commented 1 year ago

I am not very versed in cmake either.

https://github.com/Yubico/yubihsm-shell#readme lists all the packages need to build. the PKCS11 module is in the pkcs11 dir.

https://github.com/Yubico/yubihsm-shell/tree/master/pkcs11#readme talks about an yubihsm_pkcs11.conf that says how the pkcs11 module talks to the HSM.

Don't know if libcrypto can be linked static. It sounds like yubihsm_pkcs11 uses a connector https://github.com/Yubico/yubihsm-shell/blob/master/pkcs11/README.adoc#connector-configuration Has debugging settings too.

I don't have one of these, just looking at Yubico docs.

mouse07410 commented 1 year ago

Sounds to me yubiHSM2 is a non-functional pkcs11 device (unless they fix these shortcomings in their pkcs11 driver).

Very far from it.

Of course, I employ PKCS#11 with HSM only tov do what a normal person would with an already-provisioned HSM device: invoke the app to sign things (certs and docs) or decrypt (sensitive) docs. All the provisioning I do with Yubico specific tools, like yubihsm-shell.

Currently, my apps (linked against libp11 which accesses HSM via p11-kit that locates the actual module yubihsm-pkcs11.dylib) work with YubiHSM2 perfectly, doing what I described above.

If your provider cannot generate a new key on YubiHSM - I'd be the last one to complain (in fact, couldn't care less - I don't generate keys programmatically via libp11 or OpenSSL either, and don't plan to). But failure to encrypt and decrypt - that does impact my use case.

I've never linked libcrypto statically. HSM connector is configured to accept HTTPS requests from a set of whitelisted machines with appropriate certificates. I do HSM maintenance directly on the machine that runs the connector and had HSM physically plugged in.

mouse07410 commented 1 year ago

Screen log of pkcs11-provider and libp11 doing RSA encryption and decryption on YubiHSM2 device, using the same device driver and parameters/options.

$ yhsm2-rsa-encr-prov prov
YUBIHSM_PKCS11_CONF = "/Users/ur20980/yubihsm_pkcs11.conf"
YUBIHSM_PKCS11_MODULE = "/usr/local/lib/pkcs11/yubihsm_pkcs11.dylib"

Testing pkcs11-provider with YubiHSM

Generate a random data file (1000 bytes), Base64-encoded...
openssl rand -engine rdrand -base64 -out /tmp/derive.43021.text 1000
Engine "rdrand" set.

Engine "rdrand" set.
Engine "rdrand" set.
Generated
  key: ea683482d6dc3a73a1996fe97b97cb59d749d433dbfb9fe61e1b6dff17eb0bf9
  iv:  1b15df766921af00ca6d4634470e5a3e

Encrypt file /tmp/derive.43021.text with this key and IV...
openssl enc -aes-256-cfb -a -e -in /tmp/derive.43021.text -out /tmp/derive.43021.text.enc -K ea683482d6dc3a73a1996fe97b97cb59d749d433dbfb9fe61e1b6dff17eb0bf9 -iv 1b15df766921af00ca6d4634470e5a3e

Encrypting this random symmetric key to token RSA KEY MAN key...
echo ea683482d6dc3a73a1996fe97b97cb59d749d433dbfb9fe61e1b6dff17eb0bf9 | xxd -r -p -c 200 | openssl pkeyutl  -encrypt -pubin -inkey "pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;object=RSA-OAEP;type=public" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -out /tmp/derive.43021.key.enc
Could not read public key from pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;object=RSA-OAEP;type=public
80E6E348F87F0000:error:42000001:pkcs11:store_fetch:reason(1):store.c:115:Failed to get session to load keys
pkeyutl: Error initializing context

encrypted key:
xxd: /tmp/derive.43021.key.enc: No such file or directory

Decrypting the symmetric key on the token...
openssl pkeyutl  -decrypt -inkey "pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;type=private" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -in /tmp/derive.43021.key.enc | xxd -p -c 200
Could not read private key from pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;type=private
80E6E348F87F0000:error:42000001:pkcs11:store_fetch:reason(1):store.c:115:Failed to get session to load keys
pkeyutl: Error initializing context
/Users/ur20980/bin/yhsm2-rsa-encr-prov: line 103: /tmp/derive.43021.key.dec: No such file or directory

Decrypt file /tmp/derive.43021.text.enc with this key and IV...
openssl enc -aes-256-cfb -a -d -in /tmp/derive.43021.text.enc -out /tmp/derive.43021.text.dec -K  -iv 1b15df766921af00ca6d4634470e5a3e
enc: Use -help for summary.

KEY1="ea683482d6dc3a73a1996fe97b97cb59d749d433dbfb9fe61e1b6dff17eb0bf9"
KEY2=""

Decrypted key does NOT match the original!

diff: /tmp/derive.43021.text.dec: No such file or directory
Decrypted file does NOT match the original!

$ yhsm2-rsa-encr-prov
YUBIHSM_PKCS11_CONF = "/Users/ur20980/yubihsm_pkcs11.conf"
YUBIHSM_PKCS11_MODULE = "/usr/local/lib/pkcs11/yubihsm_pkcs11.dylib"

Testing pkcs11 engine (libp11) with YubiHSM

Generate a random data file (1000 bytes), Base64-encoded...
openssl rand -engine rdrand -base64 -out /tmp/derive.43038.text 1000
Engine "rdrand" set.

Engine "rdrand" set.
Engine "rdrand" set.
Generated
  key: 97638e7d562741309c42777bb0054b1874865db6b82e06975845bed02484b913
  iv:  ce214bc86cac3076d95b22abccd46d28

Encrypt file /tmp/derive.43038.text with this key and IV...
openssl enc -aes-256-cfb -a -e -in /tmp/derive.43038.text -out /tmp/derive.43038.text.enc -K 97638e7d562741309c42777bb0054b1874865db6b82e06975845bed02484b913 -iv ce214bc86cac3076d95b22abccd46d28

Encrypting this random symmetric key to token RSA KEY MAN key...
echo 97638e7d562741309c42777bb0054b1874865db6b82e06975845bed02484b913 | xxd -r -p -c 200 | openssl pkeyutl -engine pkcs11 -keyform engine -encrypt -pubin -inkey "pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;object=RSA-OAEP;type=public" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -out /tmp/derive.43038.key.enc
Engine "pkcs11" set.
Enter PKCS#11 token PIN for YubiHSM:

encrypted key:
9f9a2e08f72e98cbc85c200705e6927d8f2bface7c764e46537d595604fb733fa0c5d4eba40df4d6bbdce3a2fcf59ac0a6ad754be88159ee711dfbdbbf21f1772b1c8335cee67fb07cfd22689e83ede857af6affb98e322c6fef916cc336f8c5bfa7ec0ce067b4edc4e44048418fd8278ea648a2356c07760bddf89d1f8bf247aebd8950ec2cfddba7e9803b147bba7637ae302160e3233a44d655dc3f27e41b5b5d2b5d74c254bc03f10b8345e4628c56574d4403bb6a41d6c202dc76ab0d98e2228f4954b4d4d2
683837d2ca563df90610194c3b3e258371346fff456e66b56d79df2376823c202b7f7ecd82c8ee67fb0d0b62ffe522949df8c8af39b64a27b62718d7c88d2f16d2478bc500002eac473b71dac8a1683236fc592ce9f98c7bda68df920bb8f0bcc51ecc32eda6657b99dbd83036c7764708c9e6543a1ad4810bf5bf87e9f96c5e7525c30f1403a92e5db4fcec6a3819daa869d087e473fbb3c0a6b72036fe5723a7869a252a9fcca7bc434e94d6c3a4d3f5f020444cdff496

Decrypting the symmetric key on the token...
openssl pkeyutl -engine pkcs11 -keyform engine -decrypt -inkey "pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;type=private" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -in /tmp/derive.43038.key.enc | xxd -p -c 200
Engine "pkcs11" set.
Enter PKCS#11 token PIN for YubiHSM:

Decrypt file /tmp/derive.43038.text.enc with this key and IV...
openssl enc -aes-256-cfb -a -d -in /tmp/derive.43038.text.enc -out /tmp/derive.43038.text.dec -K 97638e7d562741309c42777bb0054b1874865db6b82e06975845bed02484b913 -iv ce214bc86cac3076d95b22abccd46d28

KEY1="97638e7d562741309c42777bb0054b1874865db6b82e06975845bed02484b913"
KEY2="97638e7d562741309c42777bb0054b1874865db6b82e06975845bed02484b913"

Original and decrypted keys  match

Original and decrypted files match

$
simo5 commented 1 year ago

Would it be possible to have a spy file of both methods for the decrypt operation ?

mouse07410 commented 1 year ago

Would it be possible to have a spy file of both methods for the decrypt operation ?

I'll do my best.

Offhand, I suspect that somehow the engine "figures" when to "outsource" an operation to another (default) provider, and pkcs11-provider doesn't yet... But let's hold off conclusions until the traces are in.

mouse07410 commented 1 year ago

Here's the SPY trace from the engine (libp11): yhsm2-eng-spy.txt

There was no SPY from provider - I suspect because it did not get to PKCS#11:

$ yhsm2-rsa-encr-prov prov

Testing pkcs11-provider with YubiHSM

YUBIHSM_PKCS11_CONF = "/Users/ur20980/yubihsm_pkcs11.conf"
YUBIHSM_PKCS11_MODULE = "/usr/local/lib/pkcs11/yubihsm_pkcs11.dylib"

Generate a random data file (1000 bytes), Base64-encoded...
openssl rand -engine rdrand -base64 -out /tmp/derive.21016.text 1000
Engine "rdrand" set.

Engine "rdrand" set.
Engine "rdrand" set.
Generated
  key: 198f39ce93077cf61d1ae3d2af886cc323e7d61ff544210cb877a672a22300c1
  iv:  3689ead336aa3b483ddc10a6fadf07a0

Encrypt file /tmp/derive.21016.text with this key and IV...
openssl enc -aes-256-cfb -a -e -in /tmp/derive.21016.text -out /tmp/derive.21016.text.enc -K 198f39ce93077cf61d1ae3d2af886cc323e7d61ff544210cb877a672a22300c1 -iv 3689ead336aa3b483ddc10a6fadf07a0

Encrypting this random symmetric key to token RSA KEY MAN key...
echo 198f39ce93077cf61d1ae3d2af886cc323e7d61ff544210cb877a672a22300c1 | xxd -r -p -c 200 | openssl pkeyutl  -encrypt -pubin -inkey "pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;object=RSA-OAEP;type=public" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -out /tmp/derive.21016.key.enc
Could not read public key from pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;object=RSA-OAEP;type=public
80E6E348F87F0000:error:42000001:pkcs11:store_fetch:reason(1):store.c:115:Failed to get session to load keys
pkeyutl: Error initializing context
simo5 commented 1 year ago

Just FYI I was finally able to test this with a (cloud) HSM:

# dd if=/dev/urandom of=/tmp/rnd bs=128 count=1
1+0 records in
1+0 records out
128 bytes copied, 0.000153159 s, 836 kB/s
# openssl pkeyutl -encrypt -pubin -inkey pkcs11:object="Generated RSA Public Key" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -in /tmp/rnd -out /tmp/rnd.enc1
# ls /tmp
debug  rnd  rnd.enc1
simo5 commented 1 year ago

So this sounds like YubiHSM specific or OpenSC related. I should be able to get my hands on a YubiHSM relatively soon, once I do I will try to figure this out.

mouse07410 commented 1 year ago

Just FYI I was finally able to test this with a (cloud) HSM:


# dd if=/dev/urandom of=/tmp/rnd bs=128 count=1

1+0 records in

1+0 records out

128 bytes copied, 0.000153159 s, 836 kB/s

# openssl pkeyutl -encrypt -pubin -inkey pkcs11:object="Generated RSA Public Key" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -in /tmp/rnd -out /tmp/rnd.enc1

# ls /tmp

debug  rnd  rnd.enc1

Interesting. Looks like this cloud HSM had more lax access control, behaving more like a normal smart card.

Out of curiosity, does it have Audit functionality? Does it allow extracting public keys without providing PIN/password?

simo5 commented 1 year ago

Heh turns out I had the pin in the configuration file, so probably does not count for much

simo5 commented 1 year ago

But then, thinking about it, if we allow to extract public keys w/o pin then we can't have our cake and eat it too ... Does your experiment work if you set the pin in the config file or provide it in the key uri ?

mouse07410 commented 1 year ago

if we allow to extract public keys w/o pin then we can't have our cake and eat it too ...

Respectfully disagree. One can always attempt to extract the pubkey without pin, and if receives an error - retry the operation asking for a pin.

Does your experiment work if you set the pin in the config file or provide it in the key uri ?

Haven't tried. Frankly, I don't even recall how to do it.

simo5 commented 1 year ago

if we allow to extract public keys w/o pin then we can't have our cake and eat it too ...

Respectfully disagree. One can always attempt to extract the pubkey without pin, and if receives an error - retry the operation asking for a pin.

This is already working, but if we extract a public key without pin, and then the next operation requires a PIN, we are out of luck, and not much I can do in the provider.

In that case the only way is to either change configuration to always request a pin (see pkcs11-module-login-behavior or pass a pin directly in configuration or in the key URI.

Does your experiment work if you set the pin in the config file or provide it in the key uri ?

Haven't tried. Frankly, I don't even recall how to do it.

Retry the command above that failed but add pin= to the key uri.

simo5 commented 1 year ago

@mouse07410 I finally have got a YubiHSM2 for testing.

I reproduced the issue, and I am pretty sure it is a driver deficiency.

Reading the documentation I see thishere:

RSA:

They explicitly say they support exclusively signing and decryption, there is no mention of Verification or Encryption.

This developer page also seem to imply a driver that does not implement all operations (ses under Capabilities and Domains): https://developers.yubico.com/YubiHSM2/Component_Reference/PKCS_11/

I am not sure why they return CKR_KEY_TYPE_INCONSISTENT instead of CKR_FUNCTION_NOT_SUPPORTED though, it is really odd, I guess I'll ask them if I can.

In the meanwhile though, this means this is expected and we should probably close this as not a bug. We can open a new feature request to try to handle in software via config quirks and/or auto-detection where possible. The fact they do not return CKR_FUNCTION_NOT_SUPPORTED makes auto-detection much harder though.

mouse07410 commented 1 year ago

They explicitly say they support exclusively signing and decryption, there is no mention of Verification or Encryption.

Yes. But that means (AFAICT) that the correct "user-friendly" behavior would be to retrieve the public key and perform the encryption in software - just like libp11 engine does with OpenSSL. After all, no physical token that I know performs encryption on board, yet the test-script works perfectly fine:

$ pkcs11-rsa-encr-prov prov

Testing pkcs11-provider

This is not a CAC
Using KEY MAN key...
Generate a random data file (1000 bytes), Base64-encoded...
openssl rand -engine rdrand -base64 -out /tmp/derive.50396.text 1000
Engine "rdrand" set.

Engine "rdrand" set.
Engine "rdrand" set.
Generated
  key: 8def572d961b6e3c0601cfb5b17b0e0d05f9adadc346395b5f8d3a8ceccfb6cc
  iv:  5279f754ff34dfb7edbe9d2661809511

Encrypt file /tmp/derive.50396.text with this key and IV...
openssl enc -aes-256-cfb -a -e -in /tmp/derive.50396.text -out /tmp/derive.50396.text.enc -K 8def572d961b6e3c0601cfb5b17b0e0d05f9adadc346395b5f8d3a8ceccfb6cc -iv 5279f754ff34dfb7edbe9d2661809511

Encrypting this random symmetric key to token RSA KEY MAN key...
echo -n 8def572d961b6e3c0601cfb5b17b0e0d05f9adadc346395b5f8d3a8ceccfb6cc | xxd -r -p -c 200 | openssl pkeyutl -engine pkcs11 -keyform engine -encrypt -pubin -inkey "pkcs11:manufacturer=piv_II;id=%03;object=KEY%20MAN%20pubkey;type=public" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -out /tmp/derive.50396.key.enc
Engine "pkcs11" set.

Decrypting the symmetric key on the token...
openssl pkeyutl  -decrypt -inkey "pkcs11:manufacturer=piv_II;id=%03;object=KEY%20MAN%20key;type=private" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -in /tmp/derive.50396.key.enc | xxd -p -c 200
Enter pass phrase for PKCS#11 Token (Slot 0 - Yubico YubiKey OTP+FIDO+CCID):

Decrypt file /tmp/derive.50396.text.enc with this key and IV...
openssl enc -aes-256-cfb -a -d -in /tmp/derive.50396.text.enc -out /tmp/derive.50396.text.dec -K 8def572d961b6e3c0601cfb5b17b0e0d05f9adadc346395b5f8d3a8ceccfb6cc -iv 5279f754ff34dfb7edbe9d2661809511

KEY1="8def572d961b6e3c0601cfb5b17b0e0d05f9adadc346395b5f8d3a8ceccfb6cc"
KEY2="8def572d961b6e3c0601cfb5b17b0e0d05f9adadc346395b5f8d3a8ceccfb6cc"

Original and decrypted keys  match

Original and decrypted files match

Cleaning up...

I am not sure why they return CKR_KEY_TYPE_INCONSISTENT instead of CKR_FUNCTION_NOT_SUPPORTED though, it is really odd, I guess I'll ask them if I can

Can't tell - probably you're right. Please do ask them, or let me know if you really prefer that I asked it. Also, perhaps @qpernil joins this conversation.

In the meanwhile though, this means this is expected and we should probably close this as not a bug.

Arguable. Especially since this problem affects all the tokens I have: Yubikey, YubiHSM2, CAC. And libp11 does not suffer from it.

We can open a new feature request to try to handle in software via config quirks and/or auto-detection where possible.

I'd claim that handling this case in software is the expected behavior, reinforced by the current PKCS#11 engine. Anything else deviates from that.

The fact they do not return CKR_FUNCTION_NOT_SUPPORTED makes auto-detection much harder though.

You have a good point here - perhaps @qpernil can help.

simo5 commented 1 year ago

Well you know what, the funny thing is that I knew OpenSSL should already try to export the public key and do the encryption operation on its own.

But I had broken exporting of keys when OpenSSL requests more than it should and yet we can fulfill the exporting.

228 fixes that and I just tested that it results in the encryption operation working.

It relies on people using a standard configuration where they allow the default provider to work and do the export/import dance. So this may break in some odd configurations and it will break if pkcs11-module-allow-export is set to 1 (which disables export).

But I think this is ok, we can think of additional fallbacks if there is any case where such a configurations and use of these HW token make sense at all.

mouse07410 commented 1 year ago

Well you know what, the funny thing is that I knew OpenSSL should already try to export the public key and do the encryption operation on its own.

Makes sense.

But I had broken exporting of keys when OpenSSL requests more than it should and yet we can fulfill the exporting.

Stuff happens. Already this provider is much better than it was, e.g., a month or so ago.

https://github.com/latchset/pkcs11-provider/pull/228 fixes that and I just tested that it results in the encryption operation working.

Great!! I've just tested it on a Yubikey, and it worked!

t relies on people using a standard configuration where they allow the default provider to work and do the export/import dance. So this may break in some odd configurations

I concur 100%.

and it will break if pkcs11-module-allow-export is set to 1 (which disables export).

My only nit here is that this parameter is confusing. To a layman it means "if you want to allow export - set this to 1". Not unlike the term "life insurance" ;-)

But I think this is ok, we can think of additional fallbacks if there is any case where such a configurations and use of these HW token make sense at all.

Absolutely. This would be the absolutely last thing I'd lose my sleep over. If a year from now a credible need arises, we can cross this bridge then. ;-)

P.S. There still are some rough edges regarding YubiHSM2. Signature test seems to fail:

$ pkcs11-rsa-sign-demo prov hsm

Testing pkcs11-provider

Testing YubiHSM2 secure device

This is not a CAC
Generating ephemeral file /tmp/derive.53029.text to test RSA-PSS signature...

openssl rand -engine rdrand -hex -out /tmp/derive.53029.text 5120
Engine "rdrand" set.

Signing file /tmp/derive.53029.text...
openssl dgst  -sign "pkcs11:token=YubiHSM;id=%04%01;type=private" -sha384 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:-1 -out /tmp/derive.53029.text.sig /tmp/derive.53029.text
Could not read private key from pkcs11:token=YubiHSM;id=%04%01;type=private

$ pkcs11-rsa-sign-demo eng hsm

Testing pkcs11 engine (libp11)

Testing YubiHSM2 secure device

This is not a CAC
Generating ephemeral file /tmp/derive.53112.text to test RSA-PSS signature...

openssl rand -engine rdrand -hex -out /tmp/derive.53112.text 5120
Engine "rdrand" set.

Signing file /tmp/derive.53112.text...
openssl dgst -engine pkcs11 -keyform engine -sign "pkcs11:token=YubiHSM;id=%04%01;type=private" -sha384 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:-1 -out /tmp/derive.53112.text.sig /tmp/derive.53112.text
Engine "pkcs11" set.
Enter PKCS#11 token PIN for YubiHSM:
Signature for /tmp/derive.53112.text is stored in /tmp/derive.53112.text.sig

Verifying signature:
openssl dgst -engine pkcs11 -keyform engine -verify "pkcs11:token=YubiHSM;id=%04%01;type=public" -sha384 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:-1 -signature /tmp/derive.53112.text.sig  /tmp/derive.53112.text
Engine "pkcs11" set.
Enter PKCS#11 token PIN for YubiHSM:
Verified OK
mouse07410 commented 1 year ago

Hmm, still does not work with YubiHSM2:

$ yhsm2-rsa-encr-prov prov
YUBIHSM_PKCS11_CONF = "/Users/ur20980/yubihsm_pkcs11.conf"
YUBIHSM_PKCS11_MODULE = "/usr/local/lib/pkcs11/yubihsm_pkcs11.dylib"

Testing pkcs11-provider with YubiHSM

Generate a random data file (1000 bytes), Base64-encoded...
openssl rand -engine rdrand -base64 -out /tmp/derive.56300.text 1000
Engine "rdrand" set.

Engine "rdrand" set.
Engine "rdrand" set.
Generated
  key: 9ac2efa2a092e622b5e06fc003da68e8779d3b8640e702695eeafa225cae5fa8
  iv:  66dc39200464190036c00a4e8bcad644

Encrypt file /tmp/derive.56300.text with this key and IV...
openssl enc -aes-256-cfb -a -e -in /tmp/derive.56300.text -out /tmp/derive.56300.text.enc -K 9ac2efa2a092e622b5e06fc003da68e8779d3b8640e702695eeafa225cae5fa8 -iv 66dc39200464190036c00a4e8bcad644

Encrypting this random symmetric key to token RSA KEY MAN key...
echo 9ac2efa2a092e622b5e06fc003da68e8779d3b8640e702695eeafa225cae5fa8 | xxd -r -p -c 200 | openssl pkeyutl  -encrypt -pubin -inkey "pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;object=RSA-OAEP;type=public" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -out /tmp/derive.56300.key.enc
Could not read public key from pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;object=RSA-OAEP;type=public
pkeyutl: Error initializing context
simo5 commented 1 year ago

I tested on a YubiHSM2 and it worked for me, perhaps you got the id wrong? Try to use just "pkcs11:object=RSA-OAEP;type=public" as the URI

simo5 commented 1 year ago

Signature also works for me, sounds like an incorrect pkcs11 URI in your last tests

mouse07410 commented 1 year ago

sounds like an incorrect pkcs11 URI in your last tests Try to use just "pkcs11:object=RSA-OAEP;type=public" as the URI

BTW, this all is with #228 applied

Here's the problem: it's an HSM, not a "mere" smartcard, and it has quite a few encryption/decryption and signature/verification key pairs. Which is why I'm afraid that specifying id: explicitly is a-must.

Also, I observe different behavior on Apple Silicon and Intel. For example, on Silicon:

$ yhsm2-rsa-encr-prov prov

Testing pkcs11-provider with YubiHSM

YUBIHSM_PKCS11_CONF = "/Users/ur20980/yubihsm_pkcs11.conf"
YUBIHSM_PKCS11_MODULE = "/usr/local/lib/pkcs11/yubihsm_pkcs11.dylib"

Generate a random data file (1000 bytes), Base64-encoded...
openssl rand -base64 -out /tmp/derive.53697.text 1000

Generated
  key: b2d6a1b3503dead9e5dc0b61b39b8fb29c2133fa33280a10bd2de6ce7c474105
  iv:  362ab0dfed2f2a450a4537631f3454fa

Encrypt file /tmp/derive.53697.text with this key and IV...
openssl enc -aes-256-cfb -a -e -in /tmp/derive.53697.text -out /tmp/derive.53697.text.enc -K b2d6a1b3503dead9e5dc0b61b39b8fb29c2133fa33280a10bd2de6ce7c474105 -iv 362ab0dfed2f2a450a4537631f3454fa

Encrypting this random symmetric key to token RSA KEY MAN key...
echo b2d6a1b3503dead9e5dc0b61b39b8fb29c2133fa33280a10bd2de6ce7c474105 | xxd -r -p -c 200 | openssl pkeyutl  -encrypt -pubin -inkey "pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;object=RSA-OAEP;type=public" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -out /tmp/derive.53697.key.enc
Enter pass phrase for PKCS#11 Token (Slot 0 - YubiHSM Connector 0.0.0.0):

encrypted key:
11b47f3160c15f5aebbbc2bfbabb79247ea30f6fc82ed2a66a55e46788b56415040669bec4d6023a565e9cc203bd286fd12c3fc3876af55e554e437cdb79bfe41e80303c49dc5cdd25f237d693a45374ec85b1e324f323b4554b5a0324a4181ecc7e01a95bd5af45d99892948261f184f9267194ef0e9cff1c1ddf97985bef5063ffbea739cc04ca2fffbbd0311859104363e2b172b00ac50ef064ee982dd51bdc478981090ec1db0bc0d87348ae8365d5c557ef5cacdf88f21eb337511752d86a4335fdd686a1af
45f4ff2ba8af186569811c08b023b8c1b8af5b6755f307f47005480574ed1f976830cde64574aee91541e2f54aebd63b322793ece97d88b2f792e237e9e7093ff36b3e738520744b69590ffc1a31bf624180704784ed3500c00769afc3383842a59a4657cde9a71af8475652ff7e5658c54eead4f36f736a71c9b0aa56439f0a894a2a6ccd4ff4effcdef447510a29f70c7421c10649dad147987fac94dfd70c26fae76e61a7066298924f4feed16d87c006ad1000c45843

Decrypting the symmetric key on the token...
openssl pkeyutl  -decrypt -inkey "pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;type=private" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -in /tmp/derive.53697.key.enc | xxd -p -c 200
Enter pass phrase for PKCS#11 Token (Slot 0 - YubiHSM Connector 0.0.0.0):

Decrypt file /tmp/derive.53697.text.enc with this key and IV...
openssl enc -aes-256-cfb -a -d -in /tmp/derive.53697.text.enc -out /tmp/derive.53697.text.dec -K b2d6a1b3503dead9e5dc0b61b39b8fb29c2133fa33280a10bd2de6ce7c474105 -iv 362ab0dfed2f2a450a4537631f3454fa

KEY1="b2d6a1b3503dead9e5dc0b61b39b8fb29c2133fa33280a10bd2de6ce7c474105"
KEY2="b2d6a1b3503dead9e5dc0b61b39b8fb29c2133fa33280a10bd2de6ce7c474105"

Original and decrypted keys  match
Original and decrypted files match

On the other hand, with YubiKey-4:

$ pkcs11-rsa-encr-prov prov

Testing pkcs11-provider

This is not a CAC
Using KEY MAN key...
Generate a random data file (1000 bytes), Base64-encoded...
openssl rand -base64 -out /tmp/derive.53921.text 1000

Generated
  key: 957eb5f7fb5aa33bef27672994f997af7782e95455d1632298d15af82752f967
  iv:  04376fcfca57463afce801ee8fefac2c

Encrypt file /tmp/derive.53921.text with this key and IV...
openssl enc -aes-256-cfb -a -e -in /tmp/derive.53921.text -out /tmp/derive.53921.text.enc -K 957eb5f7fb5aa33bef27672994f997af7782e95455d1632298d15af82752f967 -iv 04376fcfca57463afce801ee8fefac2c

Encrypting this random symmetric key to token RSA KEY MAN key...
echo -n 957eb5f7fb5aa33bef27672994f997af7782e95455d1632298d15af82752f967 | xxd -r -p -c 200 | openssl pkeyutl  -encrypt -pubin -inkey "pkcs11:manufacturer=piv_II;id=%03;object=KEY%20MAN%20pubkey;type=public" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -out /tmp/derive.53921.key.enc

Decrypting the symmetric key on the token...
openssl pkeyutl  -decrypt -inkey "pkcs11:manufacturer=piv_II;id=%03;object=KEY%20MAN%20key;type=private" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -in /tmp/derive.53921.key.enc | xxd -p -c 200
Enter pass phrase for PKCS#11 Token (Slot 0 - Yubico YubiKey OTP+FIDO+CCID):
Public Key operation error
405BA0FA01000000:error:42000070:pkcs11:p11prov_rsaenc_decrypt:An invalid mechanism was specified to the cryptographic operation:asymmetric_cipher.c:285:Failed to open session on slot 0

P.S. On several of my Mac boxes basic-softhsm test fails with

FAIL: basic-softhsm
===================

Executing ./tbasic

## Raw Sign check error
openssl pkeyutl -sign -inkey "${BASEURI}" -pkeyopt pad-mode:none -in ${TMPPDIR}/64Brandom.bin -out ${TMPPDIR}/raw-sig.bin
Public Key operation error
405BA0FA01000000:error:0200007A:rsa routines:p11prov_sig_operate:data too small for key size:signature.c:866:
qpernil commented 1 year ago

I'd be happy to help here but I need a little help from you guys asking a little more specifically as the discussion here is mostly only indirectly about yubihsm_pkcs11.

"I am not sure why they return CKR_KEY_TYPE_INCONSISTENT instead of CKR_FUNCTION_NOT_SUPPORTED though, it is really odd" can you tell me what pkcs#11 function this is referring to ? CKR_KEY_TYPE_INCONSISTENT is generally returned when the function as such would normally work, but it is currently being provided with the wrong type of key.

In cases where there are errors from yubihsm_pkcs11 it would help to turn on debugging in yubihsm_pkcs11.conf by adding a line saying 'debug'

Thanks.

qpernil commented 1 year ago

I'd also like to point out two things: The statement that the YuibiHSM doesn't perform public key operations is true for the device itself, but yubihsm_pkcs11 will perform such operations in software (by loading the public key from the HSM, which is capable of exporting the public key parts from a private key object). The other is that we have now implemented the concept of metadata, whereby it is able to manage CKA_ID and CKA_LABEL as modifiable attributes through yubihsm_pkcs11, and if needed an extra opaque data object is created to hold these attributes. This makes yubihsm_pkcs11 much more compatible with applications that store large attributes, or needs to change them after key creation for example. This functionality is entirely within yubihsm_pkcs11, the YubiHSM itself is not aware of it. The net result is that more applications can now also generate or import keys via yubihsm_pkcs11.

qpernil commented 1 year ago

And a another thing is that yubihsm_pkcs11 will project private key objects in the HSM as both private and a public key objects in yubihsm_pkcs11, for compatibility. This projection is not perfect, it currently always shows the public key even if the application didn't actually create a public key object (such as when importing a private key). We have considered adding a flag to the metadata to indicate whether the public key should be presented or not, based on whether the application created it. Currently we have not done as that would imply having to create metadata objects more often.

simo5 commented 1 year ago

@qpernil the function that returns CKR_KEY_TYPE_INCONSISTENT is C_EncryptInit() when called to encrypt with and RSA Key using RSA-OAEP.

[interface.gen.c:253] p11prov_GetSessionInfo(): Calling C_GetSessionInfo
[interface.gen.c:507] p11prov_EncryptInit(): Calling C_EncryptInit
[interface.gen.c:511] p11prov_EncryptInit(): Error: 0x00000063; Error returned by C_EncryptInit

I was able to generate keys as well (this RSA key was generted via openssl genpkey) however I got an error there reading back attributes during generation that makes the openssl command end with an apparent failure. I haven't debugged yet.

As long as getting a public key object will work, projecting is fine. But if this means the operation of getting a public key fails, this is really bad, because I have code to derive the public key if the object is not available, but if it is available and it fails to operate I have no recourse.

simo5 commented 1 year ago

I can't see where this happen in the source code, but I had installed from the tarballs made available on yubico site, perhaps they are oldish and you should try to build from the github tree?

qpernil commented 1 year ago

C_EncryptInit with a RSA public key object should work. I think we need debugging output to dig down why this happens. We have tests for this, so something unexpected must be going on.

simo5 commented 1 year ago

Here's the problem: it's an HSM, not a "mere" smartcard, and it has quite a few encryption/decryption and signature/verification key pairs. Which is why I'm afraid that specifying id: explicitly is a-must.

@mouse07410 just to be clear, I have a YubiHSM2 FIPS now and I tested this and it works.

I was suggesting dropping Id as a test, I did test with ID as well and it works, but it is tedious to use it to specify keys for tests, however here is additional evidence:

# openssl dgst  -sign "pkcs11:token=YubiHSM;object=RSA-OAEP;type=private;id=%45%54%2c%4a%7e%44%c0%aa%69%57%d8%9c%4c%16%00%90" -sha384 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:-1 -out /tmp/debug.log.sig /tmp/debug.log 
# ls -al /tmp/debug.log.sig
-rw-r--r--. 1 root root 256 Apr 14 13:02 /tmp/debug.log.sig
simo5 commented 1 year ago

And:

# openssl pkeyutl -encrypt -pubin -inkey "pkcs11:object=RSA-OAEP;type=public;id=%45%54%2c%4a%7e%44%c0%aa%69%57%d8%9c%4c%16%00%90" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -in /tmp/key.bin -out /tmp/key.enc1
# ls -al /tmp/key.enc1 
-rw-r--r--. 1 root root 256 Apr 14 13:04 /tmp/key.enc1
simo5 commented 1 year ago

@mouse07410 wrt the issue you have on mac boxes ... I have no clue, we have mac os tests in CI and they pass, so it works there. I'd have to find a brave soul with a mac that can reproduce and debug the issue to figure out what is going on.

qpernil commented 1 year ago

Can I ask what the problem was, just so I know if there was anything wrong with yubihsm_pkcs11

simo5 commented 1 year ago

@qpernil the problem was that encryption with RSA-OAEP fails when asking yubihsm_pkcs11 to do it. It fails at C_EncryptionInit with CKR_KEY_TYPE_INCONSISTENT. The bug has been closed because changing how pkcs11-provider operates now means that public key encryption operations are not even passed down to the pkcs11 driver now, they are done in openssl.

However you can keep testing this by simply adding pkcs11-module-allow-export = 1 in openssl.cnf which will deny exporting public keys to openssl and keep all operations strictly on the token.

I wonder if the problem here is that the key generation failure ended up storing only the private key somehow, and the HSM faking the public key result in it returning a private key object handle when we clearly ask for a type=public (ie class CKO_PUBLIC_KEY) and that causes later the C_EncryptInit to return this error because it expects a public key and it gets an object handle that internally resolves to a private key.

If this is the case, I call it a yubihsm_pkcs11 bug because I have no way to tell as a user that the object can't be used.

qpernil commented 1 year ago

OK, thats strange, we support that. I will investigate if there are any bugs.

simo5 commented 1 year ago

So this is an example I just went through, this time generating the key via yubihsm-shell:

yubihsm> generate asymmetric 1 0x5253 RSA-OAEP-2 1 sign-pkcs,sign-pss,decrypt-pkcs,decrypt-oaep,sign-attestation-certificate rsa2048
Generated Asymmetric key 0x5253

# openssl storeutl -keys -text pkcs11://
[...]
Key ID:
    52:53
Label: RSA-OAEP-2

# openssl pkeyutl -encrypt -pubin -inkey "pkcs11:object=RSA-OAEP-2;type=public" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -in /tmp/key.bin -out /tmp/key.enc2
Public Key operation error
00D24220407F0000:error:40800063:pkcs11:p11prov_EncryptInit:The specified key is not the correct type of key to use with the specified mechanism:interface.gen.c:511:Error returned by C_EncryptInit
simo5 commented 1 year ago

Tested with this version: yubihsm-shell-2.4.0-1.fc37.x86_64

qpernil commented 1 year ago

See https://github.com/Yubico/yubihsm-shell/pull/330, should fix this. It was a regression, this used to work. Sorry about that.

simo5 commented 1 year ago

Nice, I will try to find some time to build an updated yubihsm-shell next week on the system and test.

simo5 commented 1 year ago

Interestingly I tested signatures and while the card to perform verification it seem it always fail (but openssl also fails to verify the generate signature). I'll track this in a different bug after I determine what's the issue

simo5 commented 1 year ago

Seems to be specific to the openssl dgst command, I tested with openssl pkeyutl and all works fine

mouse07410 commented 1 year ago

Seems to be specific to the openssl dgst command, I tested with openssl pkeyutl and all works fine

@simo5 One important difference between the two is that openssl dgst runs hash over plaintext before passing request to signer, while openssl pkeyutl only puts the appropriate "wrapper", but expects that whatever hash is mentioned in the -pkeyopt arguments, has been already applied to the input.

Can this explain the error you're seeing?

mouse07410 commented 1 year ago

@qpernil the following is related to this ticket. Somehow yubihsm-shell seems to have a problem locating and returning keys.

This is with libp11 engine:

Encrypting this random symmetric key to token RSA KEY MAN key...
echo 5d6c5542f974d10cba1cc487b9a5dc22f0ca26b68a97bfb864bb92a41420559f | xxd -r -p -c 200 | openssl pkeyutl -engine pkcs11 -keyform engine -encrypt -pubin -inkey "pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;object=RSA-OAEP;type=public" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -out /tmp/derive.69616.key.enc
Engine "pkcs11" set.
Enter PKCS#11 token PIN for YubiHSM:

Decrypting the symmetric key on the token...
openssl pkeyutl -engine pkcs11 -keyform engine -decrypt -inkey "pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;object=RSA_OAEP;type=private" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -in /tmp/derive.69616.key.enc | xxd -p -c 200
Engine "pkcs11" set.
Enter PKCS#11 token PIN for YubiHSM:
The private key was not found at: pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;object=RSA_OAEP;type=private
PKCS11_get_private_key returned NULL
Could not read private key from org.openssl.engine:pkcs11:pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;object=RSA_OAEP;type=private
40E38760F87F0000:error:47800065:pkcs11 engine:ERR_ENG_error:object not found:eng_back.c:894:
40E38760F87F0000:error:13000080:engine routines:ENGINE_load_private_key:failed loading private key:crypto/engine/eng_pkey.c:79:
pkeyutl: Error initializing context

Note that the URI for private and public keys is the same (pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;object=RSA-OAEP;), the only difference being type=private and type=public correspondingly. Above (with the engine) it fails to locate the private key.

And with pkcs11-provider, it fails to even get the public key:

Encrypting this random symmetric key to token RSA KEY MAN key...
echo c80b0c4d12cb7e93a5dbfb05c61410914e1c42bbd215cb855dca3180e58f0957 | xxd -r -p -c 200 | openssl pkeyutl  -encrypt -pubin -inkey "pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;object=RSA-OAEP;type=public" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -out /tmp/derive.70028.key.enc
Could not read public key from pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;object=RSA-OAEP;type=public
pkeyutl: Error initializing context

@simo5 I thought we talked about this - first try to read the public key without prompting for PIN, and it it fails - prompt for the PIN and repeat the requested operation. Considering that some environments have more than one PKCS#11 token. E.g, I have minimum two, usually three hardware tokens (CAC, YubiKey, YubiHSM2), plus a bunch of soft-tokens:

$ p11tool --list-tokens
Token 0:
    URL: pkcs11:model=p11-kit-trust;manufacturer=PKCS%2311%20Kit;serial=1;token=Default%20Trust
    Label: Default Trust
    Type: Trust module
    Flags: uPIN uninitialized
    Manufacturer: PKCS#11 Kit
    Model: p11-kit-trust
    Serial: 1
    Module: p11-kit-trust.so

Token 1:
    URL: pkcs11:model=p11-kit-trust;manufacturer=PKCS%2311%20Kit;serial=1;token=System%20Trust
    Label: System Trust
    Type: Trust module
    Flags: uPIN uninitialized
    Manufacturer: PKCS#11 Kit
    Model: p11-kit-trust
    Serial: 1
    Module: p11-kit-trust.so

Token 2:
    URL: pkcs11:model=PKCS%2315%20emulated;manufacturer=piv_II;serial=xxxxxxxxxxx;token=XXXXXXXXXXXXX%29
    Label: xxxxxxxxxxx
    Type: Hardware token
    Flags: RNG
    Manufacturer: piv_II
    Model: PKCS#15 emulated
    Serial: xxxxxxxxxxxxx
    Module: /Library/OpenSC/lib/opensc-pkcs11.so

Token 3:
    URL: pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=e2f49c07fc2bf221;token=Botan%20PKCS%2311%20tests
    Label: Botan PKCS#11 tests
    Type: Generic token
    Flags: RNG, Requires login
    Manufacturer: SoftHSM project
    Model: SoftHSM v2
    Serial: e2f49c07fc2bf221
    Module: /opt/local/lib/softhsm/libsofthsm2.so

Token 4:
    URL: pkcs11:model=YubiHSM;manufacturer=Yubico%20%28www.yubico.com%29;serial=xxxxxxxxx;token=YubiHSM
    Label: YubiHSM
    Type: Hardware token
    Flags: RNG, Requires login
    Manufacturer: Yubico (www.yubico.com)
    Model: YubiHSM
    Serial: xxxxxxxxxxxx
    Module: /usr/local/lib/pkcs11/yubihsm_pkcs11.dylib

Token 5:
    URL: pkcs11:model=YubiKey%20YK4;manufacturer=Yubico%20%28www.yubico.com%29;serial=xxxxxxxxxxxxxx;token=YubiKey%20PIV%20%xxxxxxxxxxx
    Label: YubiKey PIV #xxxxxxxxxx
    Type: Hardware token
    Flags: RNG, Requires login
    Manufacturer: Yubico (www.yubico.com)
    Model: YubiKey YK4
    Serial: 7444666
    Module: /usr/local/lib/libykcs11.dylib
mouse07410 commented 1 year ago

@simo5 I wonder what kind of key ids you have (id=%45%54%2c%4a%7e%44%c0%aa%69%57%d8%9c%4c%16%00%90???)- they seem way out of the range. In my cases, ids are like id=%04%02 - two bytes, not 16.

I was suggesting dropping Id as a test,

I heard you - but if I drop ID, how would the code figure which of the multiple available keys to use? And how would I ensure that the key selected for, e.g., encryption, matches its counterpart when decryption key is requested?

I did test with ID as well and it works, but it is tedious to use it to specify keys for tests, however here is additional evidence:

# openssl dgst  -sign "pkcs11:token=YubiHSM;object=RSA-OAEP;type=private;id=%45%54%2c%4a%7e%44%c0%aa%69%57%d8%9c%4c%16%00%90" -sha384 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:-1 -out /tmp/debug.log.sig /tmp/debug.log 
# ls -al /tmp/debug.log.sig
-rw-r--r--. 1 root root 256 Apr 14 13:02 /tmp/debug.log.sig

Something's definitely wrong above - how can you sign with RSA-OAEP key that's supposed to be used for encryption/decryption??? @qpernil is that kosher? I thought YubiHSM2 enforces key usage?