Softhsm and PKCS11 OpenSSL engine not working

[GabrielWQS]

Hello,

I need to perform an HTTP communication with mTLS in dotnet8 and the private key must be present in a HSM.

In a Unix environment, dotnet8 should be able to access the key handle through the interface available here.

Initially, I am conducting some tests in a Docker environment using Alpine 3.18 + SoftHSM2 + OpenSSL + libp11, however, operations with the private key are not working.

My dockerfile

FROM mcr.microsoft.com/dotnet/sdk:8.0-alpine3.18

USER root

RUN apk update && \
    apk add sudo gnutls-utils softhsm pkgconf build-base perl libressl-dev linux-headers 

#https://ftp.openssl.org/source/old/1.1.1/
COPY packages/openssl-1.1.1w.tar.gz /home
RUN cd /home && tar -xf openssl-1.1.1w.tar.gz
RUN cd /home/openssl-1.1.1w && ./config && make && make install
RUN rm -rf /home/openssl-1.1.1w && rm /home/openssl-1.1.1w.tar.gz

#https://github.com/OpenSC/libp11/releases/tag/libp11-0.4.12
COPY packages/libp11-0.4.12.tar.gz /home
RUN cd /home && tar -xf libp11-0.4.12.tar.gz
RUN cd /home/libp11-0.4.12 && ./configure && make && make install
RUN rm -rf /home/libp11-0.4.12 && rm /home/libp11-0.4.12.tar.gz

# I tested the pkcs11 engine configuration by file and command line, but I didn't see any difference.
#by command
# RUN openssl engine dynamic -pre "SO_PATH:/usr/local/lib/engines-1.1/libpkcs11.so" -pre "ID:pkcs11" -pre "LIST_ADD:1" -pre "LOAD" -pre "MODULE_PATH:/usr/lib/softhsm/libsofthsm2.so"
#by file https://github.com/OpenSC/libp11?tab=readme-ov-file#pkcs-11-module-configuration
COPY packages/openssl.cnf /etc/ssl/
RUN dos2unix /etc/ssl/openssl.cnf

ENV SOFTHSM2_PIN=1111 \
    SOFTHSM2_SO_PIN=1111 \
    PARTITION=Partition

RUN softhsm2-util --init-token --slot 0 --label $PARTITION --pin $SOFTHSM2_PIN --so-pin $SOFTHSM2_SO_PIN

ENTRYPOINT ["/bin/sh"]

Running the image, I executed the following commands to create the key pair and confirm the configuration:

/ # p11tool --provider /usr/lib/softhsm/libsofthsm2.so --login --generate-rsa --bits 2048 --label test-key
warning: no --outfile was specified and the generated public key will be printed on screen.
Generating an RSA key...
Token 'Partition' with URL 'pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=3e1d752923520500;token=Partition' requires user PIN
Enter PIN:
-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAySO5WEJYAAV+S3nCdx0S
EiV9ygGbMcv5c15YVzkp5Pa93ORtBIVotN5znQiEN5zU1E0qQxotRWfE3e94V3Oi
mvHanZb/PyJdtQ6Ss8IHyjP2sBMPRj4bdMeQRjlL9Z88eCTLhY24X9RqkN1k6uOS
dKAuT7wc6olKuvYb/3u7DkOm5j9fv4FL2zZWn/wvp+dbeZEK2du1+1YrAyF1tWMd
jCLsOuT1I8qZ30JJqAmDuiJ39fCskgwoAKoeeDwkeGRPg+LlMK7rN6LcWH+ng3zQ
BnYl/MDTIZHPK6L2jAssdNmsC5YGkm4hAheRbjARdX3ywHvG5BEATJD++gJJZ1fr
LwIDAQAB
-----END PUBLIC KEY-----
/ # p11tool --login --provider=/usr/lib/softhsm/libsofthsm2.so --set-pin=1111 --list-all
Object 0:
        URL: pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=3e1d752923520500;token=Partition;id=%0B%7A%EE%A3%D7%BA%09%C7%53%A8%DA%79%D7%42%6C%37%8D%F7%64%99;object=test-key;type=public
        Type: Public key (RSA-2048)
        Label: test-key
        Flags: CKA_WRAP/UNWRAP; 
        ID: 0b:7a:ee:a3:d7:ba:09:c7:53:a8:da:79:d7:42:6c:37:8d:f7:64:99

Object 1:
        URL: pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=3e1d752923520500;token=Partition;id=%0B%7A%EE%A3%D7%BA%09%C7%53%A8%DA%79%D7%42%6C%37%8D%F7%64%99;object=test-key;type=private
        Type: Private key (RSA-2048)
        Label: test-key
        Flags: CKA_WRAP/UNWRAP; CKA_PRIVATE; CKA_NEVER_EXTRACTABLE; CKA_SENSITIVE; 
        ID: 0b:7a:ee:a3:d7:ba:09:c7:53:a8:da:79:d7:42:6c:37:8d:f7:64:99
/ # openssl engine pkcs11 -t -c
(pkcs11) pkcs11 engine
 [RSA, rsaEncryption, id-ecPublicKey]
     [ available ]

However, in a test with the private key the following error is returned:

/ # echo -n "Hello, world!" > /tmp/test.txt
/ # openssl dgst -engine pkcs11 -sign "pkcs11:type=private;object=test-key" -sha256 -keyform engine -sigopt rsa_padding_mode:pkcs1 -out /tmp/test.sig -binary /tmp/test.txt
engine "pkcs11" set.
Unable to load module (null)
PKCS11_get_private_key returned NULL
cannot load key file from engine
139628050086728:error:81065401:libp11:pkcs11_CTX_load:Unable to load PKCS#11 module:p11_load.c:77:
139628050086728:error:8006B067:pkcs11 engine:func(107):invalid parameter:eng_back.c:603:
139628050086728:error:26096080:engine routines:ENGINE_load_private_key:failed loading private key:crypto/engine/eng_pkey.c:77:
unable to load key file

What am I missing?

[GabrielWQS]

By the way, this type of test was done in the following issues #249 and dotnet rep 92798

[dwmw2]

When you run p11tool you are giving the --provider option. If the SoftHSM provider is properly installed (with a p11-kit module file) then you shouldn't need to do that. What does p11-kit list-modules or p11tool --list-tokens say?

If you haven't got things installed correctly, that could explain why the engine doesn't find them.

[dwmw2]

Unable to load module (null)

Looking closer, it looks like you built your own engine and maybe it couldn't find the p11-kit-proxy.so default module when you built it, so it can't even load that? Install p11-kit before configuring and building libp11.

[GabrielWQS]

Without including the --provider option still worked:

/ # p11tool --login --generate-rsa --bits 2048 --label test-key
warning: no --outfile was specified and the generated public key will be printed on screen.
Generating an RSA key...
Token 'Partition' with URL 'pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=53f1c2f2e0e38334;token=Partition' requires user PIN
Enter PIN:
-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAsxGmffDwMdcdS+p5XsUP
lZ2WUTvu+DZlf9sw+bFzoLH9H1dVzXnbjv+YeDcF8azQ9wEiNBQU6/T8cDL8Yc5M
SWtx6u56mVO09EnZ1sp7kLFDg4SBnBPO+UAthr6thm2i35LxMpe78eWNqQUrYj+o
WOWdMDeXFxeRYIgAsZwhMGbkYYpcoU8AAjAgrX6MnBySVLIftghQviARTLmSwQG2
9qUqaKKTCZd6DTvQ1D79RmCyCAMAydJmWTOXLsV/nsrxZIbP6cNPtY4VBFrP3hEn
FlNu72r2nHGhtWa+9eRbXlpaOZl40XL0b0eu+QLEUekf8izuRlEvh587D7U5qHEe
2QIDAQAB
-----END PUBLIC KEY-----
/ # p11tool --login --set-pin=1111 --list-all
Object 0:
        URL: pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=53f1c2f2e0e38334;token=Partition;id=%47%8E%C6%51%7A%7E%20%85%07%D7%6E%DA%5A%55%B1%F1%DB%52%9F%5D;object=test-key;type=private
        Type: Private key (RSA-2048)
        Label: test-key
        Flags: CKA_WRAP/UNWRAP; CKA_PRIVATE; CKA_NEVER_EXTRACTABLE; CKA_SENSITIVE;
        ID: 47:8e:c6:51:7a:7e:20:85:07:d7:6e:da:5a:55:b1:f1:db:52:9f:5d

Object 1:
        URL: pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=53f1c2f2e0e38334;token=Partition;id=%47%8E%C6%51%7A%7E%20%85%07%D7%6E%DA%5A%55%B1%F1%DB%52%9F%5D;object=test-key;type=public 
        Type: Public key (RSA-2048)
        Label: test-key
        Flags: CKA_WRAP/UNWRAP;
        ID: 47:8e:c6:51:7a:7e:20:85:07:d7:6e:da:5a:55:b1:f1:db:52:9f:5d

And executing the commands the return was:

/ # p11-kit list-modules
softhsm2: /usr/lib/softhsm/libsofthsm2.so
    library-description: Implementation of PKCS11
    library-manufacturer: SoftHSM
    library-version: 2.6
    token: Partition
        manufacturer: SoftHSM project
        model: SoftHSM v2
        serial-number: 53f1c2f2e0e38334
        hardware-version: 2.6
        firmware-version: 2.6
        flags:
               rng
               login-required
               user-pin-initialized
               restore-key-not-needed
               token-initialized
    token:
        manufacturer: SoftHSM project
        model: SoftHSM v2
        serial-number:
        hardware-version: 2.6
        firmware-version: 2.6
        flags:
               rng
               login-required
               restore-key-not-needed
               so-pin-locked
               so-pin-to-be-changed
/ # p11tool --list-tokens
Token 0:
        URL: pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=53f1c2f2e0e38334;token=Partition
        Label: Partition
        Type: Generic token
        Flags: RNG, Requires login
        Manufacturer: SoftHSM project
        Model: SoftHSM v2
        Serial: 53f1c2f2e0e38334
        Module: /usr/lib/softhsm/libsofthsm2.so

Looking closer, it looks like you built your own engine and maybe it couldn't find the p11-kit-proxy.so default module when you built it, so it can't even load that? Install p11-kit before configuring and building libp11.

Actually I haven't created any engine, I just intend to use OpenSSL itself to ensure that everything is configured correctly. Wouldn't just configuring openssl.cnf be enough base on README?

I also added the installation of p11-kit in the image, but the result was the same.

[dwmw2]

Actually I haven't created any engine, I just intend to use OpenSSL itself to ensure that everything is configured correctly.

But... you showed a transcript from where you built libp11 and installed it, and now you have /usr/local/lib/engines-1.1/libpkcs11.so

Is that not the engine that you're using when it reports:

Unable to load module (null)

I believe that (null) is where it should have detected the location of p11-kit-proxy.so at build time, but didn't because p11-kit wasn't present when you (or your distribution) built the engine.

[GabrielWQS]

The p11-kit package does not contain the p11-kit-proxy.so... this is only present in the p11-kit-dev. I also adjusted the path to to /usr/local/lib/engines-1.1/pkcs11.so inside the openssl.cnf and it worked.

Thank you!