Ubuntu 20LTS + SoftHSM + p11-kit + OpenSSL pkcs11 doesn't work

I can't get p11-kit configured work with the OpenSSL pkcs11 engine.

Using PKCS11_MODULE_PATH or changing openssl.cnf works.

export PKCS11_MODULE_PATH=/usr/lib/x86_64-linux-gnu/softhsm/libsofthsm2.so
openssl s_client -connect <mytlsserver> -cert ~/test/dev1-ecc_cert.pem -keyform engine -engine pkcs11 -key 'pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=861c95e903f6cc1e;token=test-token;id=%A0%00;object=test-privkey;type=private?pin-value=1234'

# Works

openssl s_client -connect <mytlsserver> -cert ~/test/dev1-ecc_cert.pem -keyform engine -engine pkcs11 -key 'pkcs11:object=test-privkey;type=private?pin-value=1234'

# Works

export PKCS11_MODULE_PATH=

# Using either PKCS11 URIs above:

engine "pkcs11" set.
Unable to load module (null)
Unable to load module (null)
PKCS11_get_private_key returned NULL
cannot load client certificate private key file from engine
140540627772736:error:81065401:libp11:pkcs11_CTX_load:Unable to load PKCS#11 module:p11_load.c:79:
140540627772736:error:26096080:engine routines:ENGINE_load_private_key:failed loading private key:../crypto/engine/eng_pkey.c:77:
unable to load client certificate private key file

Here's how I've configured this on an Ubuntu 20 LTS Server VM:

sudo apt install -y softhsm2 opensc
sudo usermod -aG softhsm "$(id -un)"

# libp11-0.4.11 compiled from sources

I've created a token and imported a private key:

softhsm2-util --init-token --slot 0 --label test-token --pin 1234 --so-pin 4321
# Convert key from PKCS#1 to PKCS#8
openssl pkcs8 -topk8 -inform PEM -outform PEM -nocrypt -in dev1-ecc_key.pem -out dev1-ecc_key.p8
# Import key within the slot
softhsm2-util --pin 1234 --import ./dev1-ecc_key.p8 --token test-token --id a000 --label test-privkey

Some other potentially relevant information:

p11tool --list-tokens
Token 0:
        URL: pkcs11:model=p11-kit-trust;manufacturer=PKCS%2311%20Kit;serial=1;token=System%20Trust
        Label: System Trust
        Type: Trust module
        Flags: uPIN uninitialized
        Manufacturer: PKCS#11 Kit
        Model: p11-kit-trust
        Serial: 1
        Module: p11-kit-trust.so

Token 1:
        URL: pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=861c95e903f6cc1e;token=test-token
        Label: test-token
        Type: Generic token
        Flags: RNG, Requires login
        Manufacturer: SoftHSM project
        Model: SoftHSM v2
        Serial: 861c95e903f6cc1e
        Module: /usr/lib/x86_64-linux-gnu/softhsm/libsofthsm2.so

p11tool --login --list-privkeys pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=861c95e903f6cc1e;token=test-token
Token 'test-token' with URL 'pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=861c95e903f6cc1e;token=test-token' requires user PIN
Enter PIN:
Object 0:
        URL: pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=861c95e903f6cc1e;token=test-token;id=%A0%00;object=test-privkey;type=private
        Type: Private key (EC/ECDSA-SECP256R1)
        Label: test-privkey
        Flags: CKA_PRIVATE; CKA_SENSITIVE;
        ID: a0:00

p11-kit list-modules
p11-kit-trust: p11-kit-trust.so
    library-description: PKCS#11 Kit Trust Module
    library-manufacturer: PKCS#11 Kit
    library-version: 0.23
    token: System Trust
        manufacturer: PKCS#11 Kit
        model: p11-kit-trust
        serial-number: 1
        hardware-version: 0.23
        flags:
               write-protected
               token-initialized
opensc-pkcs11: opensc-pkcs11.so
    library-description: OpenSC smartcard framework
    library-manufacturer: OpenSC Project
    library-version: 0.20
softhsm2: /usr/lib/x86_64-linux-gnu/softhsm/libsofthsm2.so
    library-description: Implementation of PKCS11
    library-manufacturer: SoftHSM
    library-version: 2.5
    token: test-token
        manufacturer: SoftHSM project
        model: SoftHSM v2
        serial-number: 861c95e903f6cc1e
        hardware-version: 2.5
        firmware-version: 2.5
        flags:
               rng
               login-required
               user-pin-initialized
               restore-key-not-needed
               token-initialized
    token:
        manufacturer: SoftHSM project
        model: SoftHSM v2
        serial-number:
        hardware-version: 2.5
        firmware-version: 2.5
        flags:
               rng
               login-required
               restore-key-not-needed
               so-pin-locked
               so-pin-to-be-changed

openssl engine pkcs11 -t
(pkcs11) pkcs11 engine
     [ available ]

pkcs11-tool --module /usr/lib/softhsm/libsofthsm2.so -l -p 1234 --token test-token --list-objects
Public Key Object; EC  EC_POINT 256 bits
  EC_POINT:   044104afa53e620dce6dc346828b365f5edd70819dd644c9a3fffa44c9ea7e2b58cc36ee6dc07eb64fc1c6e5f14c05729d7ac0e942a8213ddaf6607655b35097018fc9
  EC_PARAMS:  06082a8648ce3d030107
  label:      test-privkey
  ID:         a000
  Usage:      verify
  Access:     none
Private Key Object; EC
  label:      test-privkey
  ID:         a000
  Usage:      sign
  Access:     sensitive

p11-kit is loading "opensc-pkcs11: opensc-pkcs11.so" and "softhsm2: /usr/lib/x86_64-linux-gnu/softhsm/libsofthsm2.so"

libp11 engine (package libengine-pkcs11-openssl) installs /usr/lib/x86_64-linux-gnu/engines-1.1/pkcs11.so which will load the default pkcs11 module or one specified in "PKCS11_MODULE_PATH" If p11-kit is installed its pkcs11 module will be loaded and it will then load the list of modules from its list. But

export PKCS11_MODULE_PATH=/usr/lib/x86_64-linux-gnu/softhsm/libsofthsm2.so

loads this one module.

export PKCS11_MODULE_PATH=

will not work, it says don't load any module. You could try unset PKCS11_MODULE_PATH

OpenSC can also support softhsm by it sm-hsm driver. So in your examples:

"'pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=861c95e903f6cc1e;token=test-token;id=%A0%00;object=test-privkey;type=private?pin-value=1234'"

p11-kit will use the SoftHSM pkcs11 module. But in the second:

'pkcs11:object=test-privkey;type=private?pin-value=1234'

p11-kit will try all pkcs11 module and may return two private keys, one as seen by OpenSC and one as seen by SoftHSM.

https://datatracker.ietf.org/doc/html/rfc7512 defines the URI that can be used with PKCS11.

So choices are:

qualify the URI to select the library-manufacturer or manufacturer or other feature.
remove opensc from p11-kit if you don't want it.
modify the opensc.conf file to only support drivers you do want.
Use PKCS11_MODULE_PATH to point to specific pkcs11 module rather the p11-kit.

Are trying trying to build one of these packages from source?

Thank you @dengert for explaining! I will try again unsetting the variable in a few days (I think I already did but worth double-checking).

p11-kit will try all pkcs11 module and may return two private keys, one as seen by OpenSC and one as seen by SoftHSM.

Doesn't this mean there is a bug somewhere (p11kit or the pkcs11 engine)? I only have 2 tokens based on the p11tool --list-tokens command and only one that matches the label.

Are trying trying to build one of these packages from source?

Only libp11-0.4.11 was built from sources.

The URI in question: 'pkcs11:object=test-privkey;type=private?pin-value=1234' tells p11-kit to look at all tokens and you have not told p11-kit which slot, manufacture or token to look at so it may have to look at all the tokens. many tokens may have a matching "object=test-privkey;type=private`

You could try :'pkcs11:token=test-token;object=test-privkey;type=private?pin-value=1234' which would only find one of your tokens then only look for the object=test-privkey;type=private on that token.

But In many cases, you must provide the PIN. to even see that a private key is present and to use private key. An since the URI can be applied to every token it may try and verify the PIN with every token. So be careful you do not lock a device by giving the wrong pin to p11-kit-trust: p11-kit-trust.so, opensc-pkcs11: opensc-pkcs11.so and softhsm2: /usr/lib/x86_64-linux-gnu/softhsm/libsofthsm2.so

I am not a p11-kit expert, so you will have to play around to get it correct.

Why are you building libp11-0.4.11 from sources? You may have to specify the default location of a module. or specify the location of p11-kit. Most distros will configure p11-kit and libp11 for you.

So be careful you do not lock a device by giving the wrong pin

I didn't consider that. Thank you!

Why are you building libp11-0.4.11 from sources?

I don't have OpenSC or the pkcs#11 engine installed at all on any of my Ubuntu Server 20.04 VMs. (/usr/lib/x86_64-linux-gnu/engines-1.1/pkcs11.so: cannot open shared object file: No such file or directory)

Closing this issue as what I was trying to do isn't recommended and could end up in locking up some of the devices by trying invalid PINs.

Thanks again @dengert for all your help!

OpenSC / libp11

Ubuntu 20LTS + SoftHSM + p11-kit + OpenSSL pkcs11 doesn't work #437