search

latchset / pkcs11-provider

A pkcs#11 provider for OpenSSL 3.0+
Other
65 stars 39 forks source link

Fails to work with HSM (RSA OAEP) #196

Closed mouse07410 closed 1 year ago

mouse07410 commented 1 year ago

Describe the bug

Fails to either encrypt or decrypt.

To Reproduce

Encryption (with provider):

$ env | grep MODULE
PKCS11_PROVIDER_MODULE=/usr/local/lib/yubihsm_pkcs11.dylib
YUBIHSM_PKCS11_MODULE=/usr/local/lib/pkcs11/yubihsm_pkcs11.dylib
$ echo $MODULE
/usr/local/lib/yubihsm_pkcs11.dylib
$ cmp $MODULE $YUBIHSM_PKCS11_MODULE 
$ 
$ openssl pkeyutl  -encrypt -pubin -inkey "pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;object=RSA-OAEP;type=public" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -in /tmp/key.bin -out /tmp/key.enc
Could not read public key from pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;object=RSA-OAEP;type=public
pkeyutl: Error initializing context
$

Encryption failed - needed to prompt for the PIN, but it didn't.

For decryption (with provider):

$ openssl pkeyutl  -decrypt -inkey "pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;object=RSA-OAEP;type=private" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -in /tmp/key.enc -out /tmp/key.dec
Enter pass phrase for PKCS#11 Token (Slot 0 - YubiHSM Connector 0.0.0.0):
Segmentation fault: 11
$ ll /tmp/key.dec
-rw-r--r--  1 ur20980  wheel  32 Feb 14 10:23 /tmp/key.dec
$ cmp /tmp/key.bin /tmp/key.dec
$

Appears to have decrypted correctly before crashing OpenSSL. Observed crash:

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x0000000000000000
Exception Codes:       0x0000000000000001, 0x0000000000000000

Termination Reason:    Namespace SIGNAL, Code 11 Segmentation fault: 11
Terminating Process:   exc handler [15524]

VM Region Info: 0 is not in any region.  Bytes before following region: 4307480576
      REGION TYPE                    START - END         [ VSIZE] PRT/MAX SHRMOD  REGION DETAIL
      UNUSED SPACE AT START
--->  
      __TEXT                      100bef000-100c6f000    [  512K] r-x/r-x SM=COW  ...xec/*/openssl

Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   libsystem_pthread.dylib             0x7ff813668f63 pthread_rwlock_rdlock + 0
1   libcrypto.3.dylib                      0x1012e434f CRYPTO_THREAD_read_lock + 9
2   libcrypto.3.dylib                      0x10130662b ossl_method_store_cache_get + 59
3   libcrypto.3.dylib                      0x1012b8aa1 inner_evp_generic_fetch + 366
4   libcrypto.3.dylib                      0x1012b8912 evp_generic_fetch + 65
5   libcrypto.3.dylib                      0x1012b6307 evp_cipher_init_internal + 1389
6   libcrypto.3.dylib                      0x1012b709d EVP_EncryptInit_ex + 27
7   libyubihsm.2.4.0.dylib                 0x100e4eeba aes_encrypt_ex + 58
8   libyubihsm.2.4.0.dylib                 0x100e508c2 send_encrypted_msg + 1218
9   libyubihsm.2.4.0.dylib                 0x100e50259 yh_send_secure_msg + 89
10  libyubihsm.2.4.0.dylib                 0x100e590ab yh_util_close_session + 107
11  yubihsm_pkcs11.dylib                   0x100f29264 C_CloseSession + 372
12  pkcs11.dylib                           0x100e8ee77 p11prov_CloseSession + 87
13  pkcs11.dylib                           0x100e9b242 session_free + 194
14  pkcs11.dylib                           0x100e9934e p11prov_free_slots + 366
15  pkcs11.dylib                           0x100e9796b p11prov_ctx_free + 187
16  libcrypto.3.dylib                      0x1012e15ff ossl_provider_free + 61
17  libcrypto.3.dylib                      0x1013466ac OPENSSL_sk_pop_free + 45
18  libcrypto.3.dylib                      0x1012e2e02 provider_store_free + 63
19  libcrypto.3.dylib                      0x1012d91d6 CRYPTO_free_ex_data + 326
20  libcrypto.3.dylib                      0x1012d66d6 context_deinit + 98
21  libcrypto.3.dylib                      0x1012d6667 ossl_lib_ctx_default_deinit + 16
22  libcrypto.3.dylib                      0x1012d95ca OPENSSL_cleanup + 200
23  libsystem_c.dylib                   0x7ff813562c26 __cxa_finalize_ranges + 416
24  libsystem_c.dylib                   0x7ff813562a39 exit + 35
25  openssl                                0x100c131bb main + 594
26  dyld                                0x7ff813339310 start + 2432

Expected behavior

For encryption (showing with the engine):

$ openssl pkeyutl -engine pkcs11 -keyform engine -encrypt -pubin -inkey "pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;object=RSA-OAEP;type=public" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -in /tmp/key.bin -out /tmp/key.enc
Engine "pkcs11" set.
Enter PKCS#11 token PIN for YubiHSM:
$
$ ll /tmp/key.enc
-rw-r--r--  1 ur20980  wheel  384 Feb 14 10:21 /tmp/key.enc
$

For decryption (showing with the engine):

$ openssl pkeyutl -engine pkcs11 -keyform engine -decrypt -inkey "pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;object=RSA-OAEP;type=private" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -in /tmp/key.enc -out /tmp/key.dec
Engine "pkcs11" set.
Enter PKCS#11 token PIN for YubiHSM:
$ cmp /tmp/key.bin /tmp/key.dec
$

Operating environment (please complete the following information):

Token and application used (please complete the following information):

Additional context

Works with engine, partially works for decryption with HSM when invoked manually. Adding or removing object=RSA_OAEP; to the key URI seems to have no effect, probably better to remove it.

$ yhsm2-rsa-encr-prov 
YUBIHSM_PKCS11_CONF = "/Users/ur20980/yubihsm_pkcs11.conf"
YUBIHSM_PKCS11_MODULE = "/usr/local/lib/pkcs11/yubihsm_pkcs11.dylib"

Testing pkcs11 engine (libp11) with YubiHSM

Generate a random data file (1000 bytes), Base64-encoded...
openssl rand -engine rdrand -base64 -out /tmp/derive.1367.text 1000
Engine "rdrand" set.

Engine "rdrand" set.
Engine "rdrand" set.
Generated
  key: 47d3b66087235de5dd0680bba9a64a947e7589aa31b4f8d4eed73201cb4347ba
  iv:  3fbc1b3c023109e4e077e36e2d7baca3

Encrypt file /tmp/derive.1367.text with this key and IV...
openssl enc -aes-256-cfb -a -e -in /tmp/derive.1367.text -out /tmp/derive.1367.text.enc -K 47d3b66087235de5dd0680bba9a64a947e7589aa31b4f8d4eed73201cb4347ba -iv 3fbc1b3c023109e4e077e36e2d7baca3

Encrypting this random symmetric key to token RSA KEY MAN key...
echo 47d3b66087235de5dd0680bba9a64a947e7589aa31b4f8d4eed73201cb4347ba | xxd -r -p -c 200 | openssl pkeyutl -engine pkcs11 -keyform engine -encrypt -pubin -inkey "pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;object=RSA-OAEP;type=public" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -out /tmp/derive.1367.key.enc
Engine "pkcs11" set.
Enter PKCS#11 token PIN for YubiHSM:

encrypted key:
4a3d7d5870e683fc580814117507a2fb09eba5e34060d57dc00e9ace540a8c424638f1ab9633e21ec39349e39588efb8857ef0defb36cffb16eebc619278de4bdbd64e3d075d39dee1c137256df1d779e70d365ccb16fab604d73f0745a60f15a6c95e9a2e7dc872d0a61b088bd9eb2a2ee9120c603e5c55f66ec9f02e3f45a01706be69131e0b3deb3cd823ed46a79633dc06d11eee139b56aad37672144431f7109ad13f0c0b09d8a22478e58db3e850199bbe2751f0a0e2824094b1ae6df63c07417e6846e7e7
5e54632acb1ae0007a0af1a75d1e130cf2246eae3d255db903862422a76f844e7962412f34d9c04eb8b4a9bbfb64c50004fa0b7697b3f1360fe21329d14f45996ab18f7f03e9025d53f8242813a7376d1f29dacbd7af31fc2eb534a261023a110d7d1387e4415f986e6b683453cb11ebaa4d414397a517ba03673d3cf1e5804d73e9822afb41a03034fa7f051b9141be563424c74e181228a9c2b2695d76160b4e2e49ad42909be7527883c0e43a504da7786d211a865c16

Decrypting the symmetric key on the token...
openssl pkeyutl -engine pkcs11 -keyform engine -decrypt -inkey "pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;type=private" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -in /tmp/derive.1367.key.enc | xxd -p -c 200
Engine "pkcs11" set.
Enter PKCS#11 token PIN for YubiHSM:

Decrypt file /tmp/derive.1367.text.enc with this key and IV...
openssl enc -aes-256-cfb -a -d -in /tmp/derive.1367.text.enc -out /tmp/derive.1367.text.dec -K 47d3b66087235de5dd0680bba9a64a947e7589aa31b4f8d4eed73201cb4347ba -iv 3fbc1b3c023109e4e077e36e2d7baca3

KEY1="47d3b66087235de5dd0680bba9a64a947e7589aa31b4f8d4eed73201cb4347ba"
KEY2="47d3b66087235de5dd0680bba9a64a947e7589aa31b4f8d4eed73201cb4347ba"

Original and decrypted keys  match

Original and decrypted files match

Cleaning up...

$
simo5 commented 1 year ago

The crash seem to happen deep into libyubihsm ... which seem to link in openssl once again ... depending on the version of openssl linked in and how that is doen it may be due to symbol clashing or some other reason.

Can you ldd libyubihsm.2.4.0.dylib to see which openssl it links against? Also, if you have valgrind installed it would be nice to run the command through valgrind as it often gives a pretty good hint of what causes the carsh in the first place.

However note that this is happening during openssl deinit, so I think this is a similar problem to #187, where the yubikey library is liberally using the common openssl libctx, and it is invoking it after openssl already started deiniting it. May have no solution except for yuico fixing their library to use a non-default libctx.

As for the issue with reading the key, a pkcs11-spy trace would be nice.

simo5 commented 1 year ago

Ah the reason we are not asking for a PIN anymore is: #190 I guess the only way to solve this for something like yubiHSM, is to add a configuration option that always forces a login regardless of the kind of key being read. Either that or find out if a specific error is returned and retry to read the store after login. A SPY log would be helpful to see if we have such a clear error to look at during key search. Potentially we could always retry with login if no key is found at all ... but it would be annoying for tokens that do allow reading keys without PIN, and the only problem is the key search parameters simply match nothing ...

mouse07410 commented 1 year ago

The crash seem to happen deep into libyubihsm ... which seem to link in openssl once again ... depending on the version of openssl linked in and how that is doen it may be due to symbol clashing or some other reason.

Probably unlikely - everything built "normally" gets linked to OpenSSL-3.0.8.

Can you ldd libyubihsm.2.4.0.dylib to see which openssl it links against?

$ ll /usr/local/lib/yubihsm_pkcs11.dylib 
lrwxr-xr-x  1 root  wheel  42 Nov 20  2021 /usr/local/lib/yubihsm_pkcs11.dylib@ -> /usr/local/lib/pkcs11/yubihsm_pkcs11.dylib
$ otool -L /usr/local/lib/yubihsm_pkcs11.dylib 
/usr/local/lib/yubihsm_pkcs11.dylib:
    @rpath/yubihsm_pkcs11.dylib (compatibility version 0.0.0, current version 0.0.0)
    /opt/local/lib/libiconv.2.dylib (compatibility version 9.0.0, current version 9.1.0)
    @rpath/libyubihsm.2.dylib (compatibility version 2.0.0, current version 2.4.0)
    /opt/local/libexec/openssl3/lib/libcrypto.3.dylib (compatibility version 3.0.0, current version 3.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
$

Also, if you have valgrind installed it would be nice to run the command through valgrind as it often gives a pretty good hint of what causes the carsh in the first place.

Sorry, cannot. :-(

However note that this is happening during openssl deinit, so I think this is a similar problem to https://github.com/latchset/pkcs11-provider/issues/187, where the yubikey library is liberally using the common openssl libctx, and it is invoking it after openssl already started deiniting it. May have no solution except for yuico fixing their library to use a non-default libctx.

If we know for sure that this is the cause, perhaps you can help me crafting an issue - and I'll post it to Yubico. More often than not, they'd listen.

As for the issue with reading the key, a pkcs11-spy trace would be nice.

yhsm2-rsa-encr-spy.txt

Ah the reason we are not asking for a PIN anymore is: https://github.com/latchset/pkcs11-provider/pull/190 I guess the only way to solve this for something like yubiHSM, is to add a configuration option that always forces a login regardless of the kind of key being read.

Yeah, here's the problem:

I think it applies to any HSM, though YubiHSM is the only one I can fool around at will.

Potentially we could always retry with login if no key is found at all ...

I very much support this approach.

but it would be annoying for tokens that do allow reading keys without PIN, and the only problem is the key search parameters simply match nothing ...

You're certainly right - but, given the likelihood of such occurrences should be quite rare, I'd say let's bite the bullet and implement it this way. Make 85% of users/use-cases happy, and 15% unfortunately suffer - unpleasant but better than the reverse.

simo5 commented 1 year ago

Please check if #197 resolves your issue, I will pile a little bit more documentation i that PR, but the first patch is what you need

mouse07410 commented 1 year ago

Please check if #197 resolves your issue, I will pile a little bit more documentation i that PR, but the first patch is what you need

Alas, no:

$ openssl pkeyutl -encrypt -pubin -inkey "pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;object=RSA-OAEP;type=public" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -in /tmp/key.bin -out /tmp/key.enc1
Enter pass phrase for PKCS#11 Token (Slot 0 - YubiHSM Connector 0.0.0.0):
Public Key operation error
8096FA56F87F0000:error:42000063:pkcs11:p11prov_EncryptInit:The specified key is not the correct type of key to use with the specified mechanism:./interface.gen.c:413:Error returned by C_EncryptInit
Segmentation fault: 11

yhsm2-rsa-encr-spy2.txt

simo5 commented 1 year ago

Well it does solve the prompting problem and the spy file shows the key is found and available. So at least it solves that part.

However C_EncryptInit does not like that this key is used with CKM_RSA_PKCS_OAEP and the parameters we select, and returns CKR_KEY_TYPE_INCONSISTENT

There are three cases in which this is returned in opensc, so it is not clear to me why it does this.

If you can perform this exact operation but using the engine I would love to see a spy trace of that too. It is quite possible we are doing something wrong setting up OAEP paramters, as most of the tests use just regular PKCS1.5 padding for encryption.

mouse07410 commented 1 year ago

Well it does solve the prompting problem and the spy file shows the key is found and available. So at least it solves that part.

Yes.

However C_EncryptInit does not like that this key is used with CKM_RSA_PKCS_OAEP and the parameters we select, and returns CKR_KEY_TYPE_INCONSISTENT

Which confuses me too. @dengert, it became customary to ask you whenever we're stymied. ;-) The key is obviously RSA-OAEP:

Object 21:
    URL: pkcs11:model=YubiHSM;manufacturer=Yubico%20%28www.yubico.com%29;token=YubiHSM;id=%04%02;object=RSA-OAEP;type=private
    Type: Private key (RSA-3072)
    Label: RSA-OAEP
    Flags: CKA_PRIVATE; CKA_EXTRACTABLE; CKA_SENSITIVE; 
    ID: 04:02

Object 22:
    URL: pkcs11:model=YubiHSM;manufacturer=Yubico%20%28www.yubico.com%29;token=YubiHSM;id=%04%02;object=RSA-OAEP;type=public
    Type: Public key (RSA-3072)
    Label: RSA-OAEP
    Flags: CKA_EXTRACTABLE; 
    ID: 04:02

And the mechanisms HSM supports are

$ yhsm2-tool -M
Using slot 0 with a present token (0x0)
Supported mechanisms:
  RSA-PKCS, keySize={2048,4096}, hw, encrypt, decrypt, sign, verify
  SHA1-RSA-PKCS, keySize={2048,4096}, hw, sign, verify
  SHA256-RSA-PKCS, keySize={2048,4096}, hw, sign, verify
  SHA384-RSA-PKCS, keySize={2048,4096}, hw, sign, verify
  SHA512-RSA-PKCS, keySize={2048,4096}, hw, sign, verify
  RSA-PKCS-PSS, keySize={2048,4096}, hw, sign, verify
  SHA1-RSA-PKCS-PSS, keySize={2048,4096}, hw, sign, verify
  SHA256-RSA-PKCS-PSS, keySize={2048,4096}, hw, sign, verify
  SHA384-RSA-PKCS-PSS, keySize={2048,4096}, hw, sign, verify
  SHA512-RSA-PKCS-PSS, keySize={2048,4096}, hw, sign, verify
  RSA-PKCS-KEY-PAIR-GEN, keySize={2048,4096}, hw, generate_key_pair
  ECDSA-KEY-PAIR-GEN, keySize={224,521}, hw, generate_key_pair, EC F_P, EC OID, EC uncompressed
  SHA-1-HMAC, keySize={1,512}, hw, sign, verify
  GENERIC-SECRET-KEY-GEN, keySize={1,1024}, hw, generate
  SHA256-HMAC, keySize={1,512}, hw, sign, verify
  SHA384-HMAC, keySize={1,1024}, hw, sign, verify
  SHA512-HMAC, keySize={1,1024}, hw, sign, verify
  ECDSA, keySize={224,521}, hw, sign, verify, EC F_P, EC OID, EC uncompressed
  ECDSA-SHA1, keySize={224,521}, hw, sign, verify, EC F_P, EC OID, EC uncompressed
  ECDH1-DERIVE, keySize={224,521}, hw, derive, EC F_P, EC OID, EC uncompressed
  RSA-PKCS-OAEP, keySize={2048,4096}, hw, encrypt, decrypt
  mechtype-0xD9554204, keySize={128,256}, hw, encrypt, decrypt, wrap, unwrap
  ECDSA-SHA256, keySize={224,521}, hw, sign, verify, EC F_P, EC OID, EC uncompressed
  ECDSA-SHA384, keySize={224,521}, hw, sign, verify, EC F_P, EC OID, EC uncompressed
  ECDSA-SHA512, keySize={224,521}, hw, sign, verify, EC F_P, EC OID, EC uncompressed
  SHA-1, digest
  SHA256, digest
  SHA384, digest
  SHA512, digest

(Note hw at all of them except simple digests, and absence of "raw" RSA operations.)

If you can perform this exact operation but using the engine I would love to see a spy trace of that too. It is quite possible we are doing something wrong setting up OAEP paramters, as most of the tests use just regular PKCS1.5 padding for encryption

Unfortunately, again, there's no PKCS#11 traffic capture after authentication. I've no idea why. yhsm2-rsa-encr-engine-spy2.txt

Also, if you cloned that engine code - you can see how they set the OAEP (and PSS) parameters...

simo5 commented 1 year ago

Once again the pkcs11-spy traces with engine lacks all the actual cryptographic operations ... I wonder if it is because with public keys OpenSSL tends to export/import them and do the operation in software instead of deferring to the card ...

Would you mind running the decrypt operation through the spy, that one must be done on the card ...

dengert commented 1 year ago

Spy says pkcs11-module used by libp11 and engine /usr/local/lib/yubihsm_pkcs11.dylib

May need to ask Yubico.

the C_FindObjects did not find any objects. but was looking for a profile so cold be OK.

4 unknown Mechanisms Unknown Mechanism (d9554204) CK_VENDOR_SPECIFIC |"Y" "U" "B" 04 Unknown Mechanism (00001044) CKM_ECDSA_SHA256 Unknown Mechanism (00001045) CKM_ECDSA_SHA384 Unknown Mechanism (00001046) CKM_ECDSA_SHA512 (the spy should have mapped the 3 above.)

But trace says Enter PKCS#11 token PIN for YubiHSM:

And next is C_Finalize Something is missing.

libl11 and engine do support OAEP: https://github.com/OpenSC/libp11/blob/master/src/p11_pkey.c#L301-L311

But note not the "Label".

dengert commented 1 year ago

OpenSSL tends to export/import them and do the operation in software instead

Agree.

mouse07410 commented 1 year ago

I wonder if it is because with public keys OpenSSL tends to export/import them and do the operation in software instead of deferring to the card

I don't know, but @dengert concurs with you. And re-checking the traces, I did see retrieval of the pubkey. So it's possible.

Would you mind running the decrypt operation through the spy, that one must be done on the card ...

yhsm2-rsa-decr-prov-spy.txt

May need to ask Yubico.

Possible. But since the libp11 engine seems to weather these operations fine - would it be possible to make the provider do exactly the same thing?

simo5 commented 1 year ago

Possible. But since the libp11 engine seems to weather these operations fine - would it be possible to make the provider do exactly the same thing?

That's mostly up to OpenSSL I would say, not clear to me why it is behaving differently in this case. But then I also do not see why the operation should fail.

I checked your log and the DecryptInit operation is initialized just the same way the EncryptInit one is ... so I do not know why the opensc-pkcs11 driver is complaining. Possibly a bug masqueraded by the fact that usually with the engine this function is never exercised ...

simo5 commented 1 year ago

The spy you sent was taken using pkcs11-provider, did the decryption work correctly in this case (aside for the segmentation fault which is a separate issue)

mouse07410 commented 1 year ago

The spy you sent was taken using pkcs11-provider, did the decryption work correctly in this case (aside for the segmentation fault which is a separate issue)

Yes, it looks like the decrypted 32-byte value matched the original plaintext.

Current master with current #197 merged still fail encryption:

echo 7ac26e8e9960218e013d0931aca959588135e466dedf09a9ff8cd87eb214b1ec | xxd -r -p -c 200 | openssl pkeyutl  -encrypt -pubin -inkey "pkcs11:model=YubiHSM;token=YubiHSM;id=%04%02;object=RSA-OAEP;type=public" -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -out /tmp/derive.65014.key.enc
Enter pass phrase for PKCS#11 Token (Slot 0 - YubiHSM Connector 0.0.0.0):
Public Key operation error
8096FA56F87F0000:error:42000063:pkcs11:p11prov_EncryptInit:The specified key is not the correct type of key to use with the specified mechanism:./interface.gen.c:413:Error returned by C_EncryptInit
mouse07410 commented 1 year ago

Maybe the key (no pun intended) to dealing with all operations on public objects (certs, pubkeys, and such) is to extract them and let the software (OpenSSL) perform the computations? Perhaps, that's what libp11 does?

Update

From https://developers.yubico.com/YubiHSM2/Component_Reference/PKCS_11/

C_Encrypt and C_Verify for Asymmetric Keys are performed in software, as well as all of the C_Digest operations.

simo5 commented 1 year ago

@mouse07410 ykcs11 just uses openssl to do those operations, it does not expect the calling application to do them.

And I can't force this, as I may have mentioned earlier this is a generic pkcs11 provider, not tailored to only specific low spec modules. The idea is to allow any operation to be offloaded. For example you may have a HW accelerator for signature verification or asym encryption that you definitely may want to use (which is why there is a parameter to deny exporting even public keys to openssl).

If a pkcs11 driver is buggy we either get it fixed, or we'll try to do a custom quirk for that one.

mouse07410 commented 1 year ago

@simo5 I understand your reasoning. So, how do we deal with this problem? It looks like we got it solved for smartcards (sign/verify and encrypt/decrypt), and only HSM encrypt remains problematic? Plus, crash at the end of HSM decrypt?

Would you like me to open two new issues - for HSM encrypt, and for crash after HSM decrypt (which could impact user-written apps that use OpenSSL libraries to access HSM)?

simo5 commented 1 year ago

I think this bug is ok for the encryption issue, we need to find out what is causing it. For the crash, that's a separate issue, and I am convinced it is a problem in yubico's drivers. Nothing I can really do there, it is a crash in openssl destructor phase because yubico is using the default openssl context ...

mouse07410 commented 1 year ago

For the crash, that's a separate issue, and I am convinced it is a problem in yubico's drivers.

Reported in https://github.com/Yubico/yubihsm-shell/issues/319