Thales HSM does not work since libp11-0.4.11

Hello,

we're using 2 network HSMs by Thales, they come with the libcknfast.so PKCS#11 library. For redundancy reasons the keys are deployed to both HSMs.

With this setup, loading a public key (using ENGINE_load_public_key()) doesn't work with the latest release nor the current git master (89ccb1f097f5 ("Change bool attribute true/false names to _true/_false"))

The application fails with:

Found uninitialized token
Specified object not found
Found uninitialized token
Specified object not found
PKCS11_load_public_key returned NULL
At main.c:126:
- SSL error:FFFFFFFF80068065:pkcs11 engine:ctx_load_pubkey:object not found:
eng_back.c:954
- SSL error:26097081:engine routines:ENGINE_load_public_key:failed loading
public key: crypto/engine/eng_pkey.c:108

In the good case the output looks like this:

Found uninitialized token

...and continues to work with the extracted key.

We've bisected the problem down to commit 85a91f4502d4 ("eng_back: Search objects in all matching tokens"). BTW: bisecting worked like charm, thanks for keeping the tree compilable!

As URL (serial, id, object redacted) we're using:

pkcs11:id=%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55;object=2222_ccc-development

Even with the full URL, it doesn't work:

pkcs11:model=;manufacturer=nCipher%55Corp.%55Ltd;serial=aaaa-aaaa-aaaa;token=accelerator;id=%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55;object=2222_ccc-development

Here some p11tool output:

$ p11tool --provider=/opt/nfast/toolkits/pkcs11/libcknfast.so --list-tokens
Token 0:
        URL: pkcs11:model=;manufacturer=nCipher%55Corp.%55Ltd;serial=aaaa-aaaa-aaaa;token=accelerator
        Label: accelerator
        Type: Hardware token
        Manufacturer: nCipher Corp. Ltd
        Model: 
        Serial: aaaa-aaaa-aaaa
        Module: (null)

Token 1:
        URL: pkcs11:model=;manufacturer=nCipher%55Corp.%55Ltd;serial=bbbb-bbbb-bbbb;token=accelerator
        Label: accelerator
        Type: Hardware token
        Manufacturer: nCipher Corp. Ltd
        Model: 
        Serial: bbbb-bbbb-bbbb
        Module: (null)

$ p11tool --provider=/opt/nfast/toolkits/pkcs11/libcknfast.so --list-all "pkcs11:model=;manufacturer=nCipher%55Corp.%55Ltd;serial=aaaa-aaaa-aaaa;token=accelerator" | grep URL 
        URL: pkcs11:model=;manufacturer=nCipher%55Corp.%55Ltd;serial=aaaa-aaaa-aaaa;token=accelerator;id=%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55;object=2222_ccc-development;type=private
        URL: pkcs11:model=;manufacturer=nCipher%55Corp.%55Ltd;serial=aaaa-aaaa-aaaa;token=accelerator;id=%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55;object=2222_ccc-development;type=public

$ p11tool --provider=/opt/nfast/toolkits/pkcs11/libcknfast.so --list-all "pkcs11:model=;manufacturer=nCipher%55Corp.%55Ltd;serial=bbbb-bbbb-bbbb;token=accelerator" | grep URL
        URL: pkcs11:model=;manufacturer=nCipher%55Corp.%55Ltd;serial=bbbb-bbbb-bbbb;token=accelerator;id=%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55;object=2222_ccc-development;type=private
        URL: pkcs11:model=;manufacturer=nCipher%55Corp.%55Ltd;serial=bbbb-bbbb-bbbb;token=accelerator;id=%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55;object=2222_ccc-development;type=public

regards, Marc

Here more debug output in verbose mode:

good:

PKCS#11: Initializing the engine
Found 4 slots
Loading public key "pkcs11:id=%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55;object=2222_ccc-development"
Looking in slot -1 for key: id=5555555555555555555555555555555555555555 label=2222_ccc-development
[492971157] aaaa-aaaa-aaaa Rt1         uninitialized     (accelerator)
[492971158] aaaa-aaaa-aaaa Rt1 slot 0  no tok          
[492971159] bbbb-bbbb-bbbb Rt2         uninitialized     (accelerator)
[492971160] bbbb-bbbb-bbbb Rt2 slot 0  no tok          
Found uninitialized token
Found slot:  bbbb-bbbb-bbbb Rt2
Found token: accelerator
Found 22 public keys:
   1    id=...
   2    id=...
   3    id=...
   4    id=...
   5    id=...
   6    id=...
   7    id=...
   8    id=5555555555555555555555555555555555555555 label=2222_ccc-development
   9    id=...
  10    id=...
  11    id=...
  12    id=...
  13    id=...
  14    id=...
  15    id=...
  16    id=...
  17    id=...
  18    id=...
  19    id=...
  20    id=...
  21    id=...
  22    id=...

bad:

PKCS#11: Initializing the engine
Found 4 slots
Loading public key "pkcs11:id=%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55;object=2222_ccc-development"
Looking in slot -1 for key: id=5555555555555555555555555555555555555555 label=2222_ccc-development
[492971157] aaaa-aaaa-aaaa Rt1         uninitialized     (accelerator)
Found uninitialized token
[492971158] aaaa-aaaa-aaaa Rt1 slot 0  no tok          
[492971159] bbbb-bbbb-bbbb Rt2         uninitialized     (accelerator)
Found uninitialized token
[492971160] bbbb-bbbb-bbbb Rt2 slot 0  no tok          
Specified object not found
Loading public key "pkcs11:id=%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55%55;object=2222_ccc-development"
Looking in slot -1 for key: id=5555555555555555555555555555555555555555 label=2222_ccc-development
[492971157] aaaa-aaaa-aaaa Rt1         uninitialized     (accelerator)
Found uninitialized token
[492971158] aaaa-aaaa-aaaa Rt1 slot 0  no tok          
[492971159] bbbb-bbbb-bbbb Rt2         uninitialized     (accelerator)
Found uninitialized token
[492971160] bbbb-bbbb-bbbb Rt2 slot 0  no tok          
Specified object not found
PKCS11_load_public_key returned NULL
At main.c:136:
- SSL error:FFFFFFFF80068065:pkcs11 engine:ctx_load_pubkey:object not found: eng_back.c:954
- SSL error:26097081:engine routines:ENGINE_load_public_key:failed loading public key: crypto/engine/eng_pkey.c:108

OpenSC / libp11

Thales HSM does not work since libp11-0.4.11 #489