search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.25k stars 4.73k forks source link

How to sign data when using pkcs11 engine for openssl with a private key stored on an HSM #92798

Closed chaoshades closed 1 month ago

chaoshades commented 1 year ago

Description

Hi all,

We are working on a way to allow our dotnet api to sign some data using a private key that is stored on an on-premise NShield Connect HSM from Entrust. We are using pkcs11 to communicate with the HSM with a module (libcknfast.so) provided by the vendor which is compatible.

 We also already tested the new feature added in dotnet 8 concerning the easier way to open keys from OpenSSL that works really well.

An exception is raised when we call SignData on the RSAOpenSsl instance created from the SafeEvpPKeyHandle returned by the new dotnet 8 feature OpenPrivateKeyFromEngine.

Timeline

The issue is not critical for now, we were already planning to wait for the official release of dotnet 8 before starting to code the new api that will be using this.

We will update proactively if the status changes.

We understand full well that we have an on-premise HSM that is not easily accessible for you to test. We will make us available if there is a need to interact or experiment things.

Reproduction Steps

Using this snippet of code : 

byte[] data = ...;
using SafeEvpPKeyHandle privateKeyHandle = SafeEvpPKeyHandle.OpenPrivateKeyFromEngine("pkcs11", "pkcs11:type=private;object=my-private-key");
using RSAOpenSsl rsaPri = new(privateKeyHandle);
rsaPri.SignData(data, HashAlgorithmName.SHA256, RSASignaturePadding.Pkcs1);

Expected behavior

The data is signed using the private key stored into the HSM.

Actual behavior

rsaPri.SignData(...) raises the following exception :

Interop+Crypto+OpenSslCryptographicException: error:04084093:rsa routines::value missing
at Interop.Crypto.RsaSignHash(SafeEvpPKeyHandle pkey, RSASignaturePaddingMode paddingMode, IntPtr digestAlgorithm, ReadOnlySpan`1 hash, Span`1 destination)
at System.Security.Cryptography.RSAOpenSsl.TrySignHash(ReadOnlySpan`1 hash, Span`1 destination, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding, Boolean allocateSignature, Int32& bytesWritten, Byte[]& signature)
at System.Security.Cryptography.RSAOpenSsl.SignHash(Byte[] hash, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding)
at System.Security.Cryptography.RSA.SignData(Byte[] data, Int32 offset, Int32 count, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding)

Configuration

OS `sh-4.4# cat /etc/os-release` ``` NAME="Red Hat Enterprise Linux" VERSION="8.8 (Ootpa)" ID="rhel" ID_LIKE="fedora" VERSION_ID="8.8" PLATFORM_ID="platform:el8" PRETTY_NAME="Red Hat Enterprise Linux 8.8 (Ootpa)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" HOME_URL="https://www.redhat.com/" DOCUMENTATION_URL="https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8" BUG_REPORT_URL="https://bugzilla.redhat.com/" REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" REDHAT_BUGZILLA_PRODUCT_VERSION=8.8 REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" REDHAT_SUPPORT_PRODUCT_VERSION="8.8" ```
Dotnet `sh-4.4# dotnet --info` ``` Host: Version: 8.0.0-rc.2.23470.7 Architecture: x64 Commit: 49bf70a429 RID: linux-x64 .NET SDKs installed: No SDKs were found. .NET runtimes installed: Microsoft.AspNetCore.App 8.0.0-rtm.23471.8 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.NETCore.App 8.0.0-rc.2.23470.7 [/usr/share/dotnet/shared/Microsoft.NETCore.App] Other architectures found: None Environment variables: Not set global.json file: Not found Learn more: https://aka.ms/dotnet/info Download .NET: https://aka.ms/dotnet/download ```
OpenSSL `sh-4.4# openssl version` ``` OpenSSL 1.1.1k FIPS 25 Mar 2021 ``` `sh-4.4# openssl engine pkcs11 -t -v` ``` (pkcs11) pkcs11 engine [ available ] SO_PATH, MODULE_PATH, PIN, VERBOSE, QUIET, INIT_ARGS, FORCE_LOGIN ``` `sh-4.4# cat /etc/pki/tls/openssl.cnf` ``` openssl_conf = openssl_def [openssl_def] engines = engine_section [engine_section] pkcs11 = pkcs11_section [pkcs11_section] engine_id = pkcs11 dynamic_path = /usr/lib64/engines-1.1/pkcs11.so MODULE_PATH = /opt/nfast/toolkits/pkcs11/libcknfast.so init = 0 ```
Key [Private Key Pkcs11 Extensions](https://github.com/dotnet/runtime/files/12754317/cklist.log) `p11tool --provider /opt/nfast/toolkits/pkcs11/libcknfast.so --list-privkeys` ``` Object 0: URL: pkcs11:...object=my-private-key;type=private Type: Private key (RSA-2048) Label: my-private-key Flags: CKA_WRAP/UNWRAP; CKA_SENSITIVE; ID: ... ```

Other information

Following the above stack trace, it brings us here : https://github.com/dotnet/runtime/blob/df8cbd11b84a5930ec71666ad87479d559872718/src/native/libs/System.Security.Cryptography.Native/pal_evp_pkey_rsa.c#L254C1-L263C6

We did some trial and error to determine where the root cause was :

  1. We tried using purely OpenSSL commands.
  2. We tried using C code, that was provided to us by @krwq (greatly appreciated 🥇 )

    We made contact through our Microsoft account manager

Please note that we cut the pkcs11 uri in the following samples for brevity.

The ones we are using were provided by tools like p11tool or from the HSM-vendor tool.

Using OpenSSL

Output `echo -n "Hello, world!" > /tmp/test.txt` ``` (No output) ``` `openssl dgst -engine pkcs11 -sign "pkcs11:type=private;object=my-private-key" -sha256 -keyform engine -sigopt rsa_padding_mode:pkcs1 -out /tmp/test.sig -binary /tmp/test.txt` ``` engine "pkcs11" set. ``` `openssl dgst -engine pkcs11 -verify "pkcs11:type=public;object=my-public-key" -sha256 -keyform engine -signature /tmp/test.sig -binary /tmp/test.txt` ``` engine "pkcs11" set. Verified OK ```

Everything went well. So we believe this would eliminate potential issues that could be on our end with OpenSSL configurations and that the proprietary module is not fiddling with the process.

Using C code

Output [ossl_test.c.txt](https://github.com/dotnet/runtime/files/12754314/ossl_test.c.txt) [build.sh.txt](https://github.com/dotnet/runtime/files/12754311/build.sh.txt) `sh-4.4# ./build.sh` ``` Compilation successful. Executable is named ossl_test. ``` `sh-4.4# ./ossl_test` ``` Main - Got Private Key! Got RSA! RSA_FLAG_EXT_PKEY:32 d:1 Factors - p:1 q:1 CRT Params - dmp1:1 dmq1:1 iqmp:1 Main - HasNoPrivateKey Result:1 Signature verification succeeded. Main - Got Signature! ``` Let's explain the output a little : - `Main - Got Private Key!` means that the pkcs11 engine was initialized and the private key was loaded accordingly. - `Got RSA!` until `Main - HasNoPrivateKey...` are printf used to debug the `HasNoPrivateKey` function. > The value next to each key property (d, p, q, etc) are boolean that checks if it is NULL. > In that case, everything is expected to be NULL because the key is not exportable from the HSM. - `Main - Got Signature!` means the signature was done correctly using the pkcs11 engine.

Everything went well for the signature.

However, HasNoPrivateKey returns 1. Which means it looks like there are no private key and then dotnet will raise the exception above : error:04084093:rsa routines::value missing

We added the RSA_FLAG_EXT_PKEY flag in the output because we found an older issue that looked like ours with a different context.

We need some guidance if there is something that could be done about the HasNoPrivateKey function or if there something to do with the RSA_FLAG_EXT_PKEY flag.

We created the issue as requested by our Microsoft account manager to allow you and/or the community to have a handle on something more tangible.

ghost commented 1 year ago

Tagging subscribers to this area: @dotnet/area-system-security, @bartonjs, @vcsjones See info in area-owners.md if you want to be subscribed.

Issue Details
## Description Hi all, We are working on a way to allow our dotnet api to sign some data using a private key that is stored on an on-premise NShield Connect HSM from Entrust. We are using pkcs11 to communicate with the HSM with a module (`libcknfast.so`) provided by the vendor which is compatible. > We also already tested the new feature added in dotnet 8 concerning the [easier way to open keys from OpenSSL](https://github.com/dotnet/runtime/issues/55356) that **works really well**. An exception is raised when we call `SignData` on the `RSAOpenSsl` instance created from the `SafeEvpPKeyHandle` returned by the new dotnet 8 feature `OpenPrivateKeyFromEngine`. ### Timeline The issue is not critical for now, we were already planning to wait for the official release of dotnet 8 before starting to code the new api that will be using this. > We will update proactively if the status changes. We understand full well that we have an on-premise HSM that is not easily accessible for you to test. We will make us available if there is a need to interact or experiment things. ## Reproduction Steps Using this snippet of code : ```csharp byte[] data = ...; using SafeEvpPKeyHandle privateKeyHandle = SafeEvpPKeyHandle.OpenPrivateKeyFromEngine("pkcs11", "pkcs11:type=private;object=my-private-key"); using RSAOpenSsl rsaPri = new(privateKeyHandle); rsaPri.SignData(data, HashAlgorithmName.SHA256, RSASignaturePadding.Pkcs1); ``` ## Expected behavior The data is signed using the private key stored into the HSM. ## Actual behavior `rsaPri.SignData(...)` raises the following exception : ```text Interop+Crypto+OpenSslCryptographicException: error:04084093:rsa routines::value missing at Interop.Crypto.RsaSignHash(SafeEvpPKeyHandle pkey, RSASignaturePaddingMode paddingMode, IntPtr digestAlgorithm, ReadOnlySpan`1 hash, Span`1 destination) at System.Security.Cryptography.RSAOpenSsl.TrySignHash(ReadOnlySpan`1 hash, Span`1 destination, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding, Boolean allocateSignature, Int32& bytesWritten, Byte[]& signature) at System.Security.Cryptography.RSAOpenSsl.SignHash(Byte[] hash, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding) at System.Security.Cryptography.RSA.SignData(Byte[] data, Int32 offset, Int32 count, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding) ``` ## Configuration
OS `sh-4.4# cat /etc/os-release` ``` NAME="Red Hat Enterprise Linux" VERSION="8.8 (Ootpa)" ID="rhel" ID_LIKE="fedora" VERSION_ID="8.8" PLATFORM_ID="platform:el8" PRETTY_NAME="Red Hat Enterprise Linux 8.8 (Ootpa)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" HOME_URL="https://www.redhat.com/" DOCUMENTATION_URL="https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8" BUG_REPORT_URL="https://bugzilla.redhat.com/" REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" REDHAT_BUGZILLA_PRODUCT_VERSION=8.8 REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" REDHAT_SUPPORT_PRODUCT_VERSION="8.8" ```
Dotnet `sh-4.4# dotnet --info` ``` Host: Version: 8.0.0-rc.2.23470.7 Architecture: x64 Commit: 49bf70a429 RID: linux-x64 .NET SDKs installed: No SDKs were found. .NET runtimes installed: Microsoft.AspNetCore.App 8.0.0-rtm.23471.8 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.NETCore.App 8.0.0-rc.2.23470.7 [/usr/share/dotnet/shared/Microsoft.NETCore.App] Other architectures found: None Environment variables: Not set global.json file: Not found Learn more: https://aka.ms/dotnet/info Download .NET: https://aka.ms/dotnet/download ```
OpenSSL `sh-4.4# openssl version` ``` OpenSSL 1.1.1k FIPS 25 Mar 2021 ``` `sh-4.4# openssl engine pkcs11 -t -v` ``` (pkcs11) pkcs11 engine [ available ] SO_PATH, MODULE_PATH, PIN, VERBOSE, QUIET, INIT_ARGS, FORCE_LOGIN ``` `sh-4.4# cat /etc/pki/tls/openssl.cnf` ``` openssl_conf = openssl_def [openssl_def] engines = engine_section [engine_section] pkcs11 = pkcs11_section [pkcs11_section] engine_id = pkcs11 dynamic_path = /usr/lib64/engines-1.1/pkcs11.so MODULE_PATH = /opt/nfast/toolkits/pkcs11/libcknfast.so init = 0 ```
Key [Private Key Pkcs11 Extensions](https://github.com/dotnet/runtime/files/12754317/cklist.log) `p11tool --provider /opt/nfast/toolkits/pkcs11/libcknfast.so --list-privkeys` ``` Object 0: URL: pkcs11:...object=my-private-key;type=private Type: Private key (RSA-2048) Label: my-private-key Flags: CKA_WRAP/UNWRAP; CKA_SENSITIVE; ID: ... ```
## Other information Following the above stack trace, it brings us here : https://github.com/dotnet/runtime/blob/df8cbd11b84a5930ec71666ad87479d559872718/src/native/libs/System.Security.Cryptography.Native/pal_evp_pkey_rsa.c#L254C1-L263C6 We did some trial and error to determine where the root cause was : 1. We tried using purely OpenSSL commands. 2. We tried using C code, that was provided to us by @krwq (greatly appreciated 🥇 ) > We made contact through our Microsoft account manager Please note that we cut the pkcs11 uri in the following samples for brevity. > The ones we are using were provided by tools like p11tool or from the HSM-vendor tool. ### Using OpenSSL
Output `echo -n "Hello, world!" > /tmp/test.txt` ``` (No output) ``` `openssl dgst -engine pkcs11 -sign "pkcs11:type=private;object=my-private-key" -sha256 -keyform engine -sigopt rsa_padding_mode:pkcs1 -out /tmp/test.sig -binary /tmp/test.txt` ``` engine "pkcs11" set. ``` `openssl dgst -engine pkcs11 -verify "pkcs11:type=public;object=my-public-key" -sha256 -keyform engine -signature /tmp/test.sig -binary /tmp/test.txt` ``` engine "pkcs11" set. Verified OK ```
**Everything went well.** So we believe this would eliminate potential issues that could be on our end with OpenSSL configurations and that the proprietary module is not fiddling with the process. ### Using C code
Output [ossl_test.c.txt](https://github.com/dotnet/runtime/files/12754314/ossl_test.c.txt) [build.sh.txt](https://github.com/dotnet/runtime/files/12754311/build.sh.txt) `sh-4.4# ./build.sh` ``` Compilation successful. Executable is named ossl_test. ``` `sh-4.4# ./ossl_test` ``` Main - Got Private Key! Got RSA! RSA_FLAG_EXT_PKEY:32 d:1 Factors - p:1 q:1 CRT Params - dmp1:1 dmq1:1 iqmp:1 Main - HasNoPrivateKey Result:1 Signature verification succeeded. Main - Got Signature! ``` Let's explain the output a little : - `Main - Got Private Key!` means that the pkcs11 engine was initialized and the private key was loaded accordingly. - `Got RSA!` until `Main - HasNoPrivateKey...` are printf used to debug the `HasNoPrivateKey` function. > The value next to each key property (d, p, q, etc) are boolean that checks if it is NULL. > In that case, everything is expected to be NULL because the key is not exportable from the HSM. - `Main - Got Signature!` means the signature was done correctly using the pkcs11 engine.
**Everything went well for the signature.** **However, `HasNoPrivateKey` returns 1.** Which means it looks like there are no private key and then dotnet will raise the exception above : `error:04084093:rsa routines::value missing` > We added the `RSA_FLAG_EXT_PKEY` flag in the output because we found an [older issue](https://github.com/dotnet/runtime/issues/53345) that looked like ours with a different context. We need some guidance if there is something that could be done about the `HasNoPrivateKey` function or if there something to do with the `RSA_FLAG_EXT_PKEY` flag. > We created the issue as requested by our Microsoft account manager to allow you and/or the community to have a handle on something more tangible.
Author: chaoshades
Assignees: -
Labels: `area-System.Security`
Milestone: -
krwq commented 1 year ago

@chaoshades I've played around with SoftHSM to generate keys for OpenSC's pkcs11 engine and tpm2tss with tpm keys and I've convinced myself this is specific to your pkcs11 engine implementation and not strictly a bug in .NET. As a side note tpm2tss has some issues with RSA keys and it don't seem to work in that combination (RSA doesn't seem to be implemented yet).

I believe your pkcs11 implementation should be setting following flag on the RSA key when creating EVP_PKEY instance:

//RSA* rsa = ... (wherever you create it in your pkcs11 code)
RSA_set_flags(rsa, RSA_FLAG_EXT_PKEY);

if some older OpenSSL then:
rsa->flags |= RSA_FLAG_EXT_PKEY;

having said that the check we do seems redundant and it seems to be causing issue with at least one implementation - while it provides slightly better exception message when you accidentally pass public key for something expecting private key I'm not sure extra work and extra issue it causes is worth it (and we should probably let OpenSSL decide what's best error to display) so I'll still consider removing HasNoPrivateKey and fixing this issue in .NET.

chaoshades commented 1 year ago

@krwq We will ask our vendor if they set this flag in their module.

About the HasNoPrivateKey, if I remember correctly, this check protect from segfault. So maybe removing it would be easier to do by moving to OpenSSL 3.0 since this will be handled entirely by OpenSSL then.

jeffhandley commented 1 year ago

Based on the findings above and the anticipation that this issue can be addressed without a change to .NET 8, I'm going to move this issue to the 9.0.0 milestone so that it no longer shows up as blocking the 8.0.0 release.

If the end-to-end does still end up being blocked and we find a change is necessary into .NET 8 for the scenario to fully work, we can explore a fix into .NET 9 first with the potential to backport into .NET 8 servicing. Thanks @krwq and @chaoshades for the close collaboration here!

chaoshades commented 1 year ago

We got feedback from the vendor and the team managing the HSM and the actual pkcs11 engine implementation doesn't set the RSA_FLAG_EXT_PKEY flag. Since this flag is also deprecated in favor of OpenSSL 3.0, we kinda understand adding deprecated new things in the engine implementation would be harder.

We also believes a workaround could be possible, like a native call or something that simulate RSA_set_flags(rsa, RSA_FLAG_EXT_PKEY); that would be called before SignData to let the HasNoPrivateKey function react with the expected behavior.

However, our client doesn't want to maintain a fix over .NET 8 for production use. The expertise is quite niche and they prefer to wait rather than risk it. So, moving this issue as non-blocking the .NET 8 release is fine for us. We will continue to follow the issue because as we said above : We will make us available if there is a need to interact or experiment things.

I may experiment on the workaround in my free time, just to see if my previous idea could work.

jeffhandley commented 3 months ago

@chaoshades We implemented OpenSSL Providers support in .NET 9. Have you moved to OpenSSL 3.x by chance? It's unlikely that we'll add new OpenSSL 1.x ENGINE features.

krwq commented 3 months ago

@chaoshades I have fixed this issue in https://github.com/dotnet/runtime/pull/104961 (will be available in preview 7 or already in nightlies) but only when using OpenSSL 3.x (due to OpenSSL segfaulting on some lower versions we need to continue doing that check and RSA_FLAG_EXT_PKEY is required). For providers this issue will not show up (but that is also due to the fact that they need OpenSSL 3.x).

chaoshades commented 2 months ago

We implemented OpenSSL Providers support in .NET 9. Have you moved to OpenSSL 3.x by chance? It's unlikely that we'll add new OpenSSL 1.x ENGINE features.

Our client didn't upgrade to OpenSSL 3.x because the usage is pretty limited for now. However, we don't have any reasons not to upgrade. The vendor also have an integration guide now. So, we believe it is doable.

I have fixed this issue in #104961 (will be available in preview 7 or already in nightlies) but only when using OpenSSL 3.x (due to OpenSSL segfaulting on some lower versions we need to continue doing that check and RSA_FLAG_EXT_PKEY is required). For providers this issue will not show up (but that is also due to the fact that they need OpenSSL 3.x).

Oh nice! I will contact the team leader to let them know if they can give me some time to test this. However, I know we are looking into the Azure Key Vault service tiers with our client for the same use-case, which may be better for them in the long run. I hope this won't stop us to at least try the proposed solution though.

jeffhandley commented 1 month ago

@chaoshades I'm going to go ahead and close this issue since we're not making any further investments into OpenSSL ENGINE support. Hopefully you will be able to proceed with OpenSSL 3.x and Providers. Thanks for reaching out on this topic.