Configurable provider during signing process for TLS

[KORe4]

We want to use a HSM (hardware security module) which came with an own JCE provider. This Provider must be used during the signing process because only the HSM provider has access to the PrivateKeys. For this purpose, we need to configure the provider for the signing procedure. All TLS functionality must be handled by BCJSSE.

SSLContext pseudo code:

SSLContext sslContext = SSLContext.getInstance("TLSv1.3", BCJSSE);
KeyManager[] keyManagerList = {new HSMX509ExtendedKeyManager(HSMKeyStoreProvider)};
TrustManager[] trustManagerList = {new HSMX509ExtendedTrustManager(HSMTrustStoreProvider)};
sslContext.init(keyManagerList,  trustManagerList, null);

the following code change would be a possible solution: JcaTlsECDSA13Signer.java

public byte[] generateRawSignature(SignatureAndHashAlgorithm algorithm, byte[] hash, Provider provider)
        throws IOException
    {...}

Is this change implementable?

[peterdettman]

Normally if you use Signer.initSign with a private key it should use the right provider automatically (i.e. the one that created the private key). Is that not working for you?

[KORe4]

No, this does not work for me. The private key was generated by a CryptoServerProvider. The debugging says that BouncyCastle is used for the signing procedure.

[peterdettman]

Does your HSM provider actually implement "NoneWithECDSA"? Have you configured your HSM provider at higher priority than the BC provider?

Ideally you could debug generateRawSignature and find out why the signer creation and init doesn't resolve to your HSM provider (Signature.getInstance should be creating a Signature$Delegate which delays provider selection until the initSign call so that it can use the correct provider based on the private key).

It is possible to configure a BouncyCastleJsseProvider with a custom subclass of JcaTlsCryptoProvider, which in turn could create the JcaTlsCrypto instance using a custom subclass of e.g. DefaultJcaJceHelper which handles createSigner by using a specific provider. It's a bit verbose, but it will at least get you past the provider-selection issue and might help diagnosing why it's not working the usual way.

[Stefan4112]

A colleague of KORe4 here.

The class Signature$Delegate is used, but seems broken. At least for our use case.

If we put BC provider at highest priority it will always be selected. Then it will crash because it can not handle the HSM Keys.

If we put HSM provider at highest priority it will always be selected. Then it will crash because it can not handle the BC Keys.

Also we have multiple slots inside the HSM and each has its own provider that can not handle other slots. So again, it crashes.

Signature$Delegate seems broken because it does not check if the provider knows the class of the key. There is a check, but it never runs because the first check supportsKeyFormat will skip it.

In our current workaround we have a custom JcaJceHelper, but we must have the key to select the correct provider. So we changed it in our BC fork. Having a fork is a bad solution imho.

Can you change the interface JcaJceHelper to provide the key and certificate? Then we could use the specific class of the key to choose the provider. And in case of HSM get the information from the key to select the correct HSM provider. Or maybe our own delegate provider would be a better solution?