Support PKCS12 keystore created by openssl as truststore

[henryju]

Hi,

We are trying to get rid of the keytool need for non java users. I was assuming PKCS12 was a good format, since it was said to be a standard. Unfortunately, it seems there is some differences between openssl and keytool when creating a truststore (containing only the public key to be trusted).

Here is how I am creating the truststore with keytool:

keytool -import -alias localhost -keystore truststore-keytool.p12 -file server.pem -storetype PKCS12

Here is how I am creating the truststore with openssl:

 openssl pkcs12 -export -name localhost -nokeys -in server.pem -out truststore-openssl.p12.p12

The truststore created by OpenSSL doesn't work:

java.security.cert.CertificateException: None of the TrustManagers trust this certificate chain
    at nl.altindag.ssl.trustmanager.CombinableX509TrustManager.checkTrusted(CombinableX509TrustManager.java:61)
    at nl.altindag.ssl.trustmanager.CompositeX509ExtendedTrustManager.checkServerTrusted(CompositeX509ExtendedTrustManager.java:86)
    at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1335)
    ... 37 more
    Suppressed: java.security.cert.CertificateException: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
        at nl.altindag.ssl.trustmanager.CombinableX509TrustManager.checkTrusted(CombinableX509TrustManager.java:54)
        ... 39 more
    Caused by: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
        at java.base/java.security.cert.PKIXParameters.setTrustAnchors(PKIXParameters.java:200)
        at java.base/java.security.cert.PKIXParameters.<init>(PKIXParameters.java:120)
        at java.base/java.security.cert.PKIXBuilderParameters.<init>(PKIXBuilderParameters.java:104)
        at java.base/sun.security.validator.PKIXValidator.<init>(PKIXValidator.java:98)
        at java.base/sun.security.validator.Validator.getInstance(Validator.java:181)
        at java.base/sun.security.ssl.X509TrustManagerImpl.getValidator(X509TrustManagerImpl.java:309)
        at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrustedInit(X509TrustManagerImpl.java:183)
        at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:198)
        at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:132)
        at nl.altindag.ssl.trustmanager.DelegatingX509ExtendedTrustManager.checkServerTrusted(DelegatingX509ExtendedTrustManager.java:48)
        at nl.altindag.ssl.trustmanager.CompositeX509ExtendedTrustManager.lambda$checkServerTrusted$4(CompositeX509ExtendedTrustManager.java:86)
        at nl.altindag.ssl.trustmanager.CombinableX509TrustManager.checkTrusted(CombinableX509TrustManager.java:41)
        ... 39 more

When looking at the two keystores:

$ openssl pkcs12 -nokeys -info -in  truststore-keytool.p12
MAC: sha256, Iteration 10000
MAC length: 32, salt length: 20
PKCS7 Encrypted data: PBES2, PBKDF2, AES-256-CBC, Iteration 10000, PRF hmacWithSHA256
Certificate bag
Bag Attributes
    friendlyName: localhost
    2.16.840.1.113894.746875.1.1: <Unsupported tag 6>
subject=C = CH, ST = Geneva, L = Geneva, O = SonarSource SA, CN = localhost
issuer=C = CH, ST = Geneva, L = Geneva, O = SonarSource SA, CN = SonarSource SA
-----BEGIN CERTIFICATE-----
[....]
-----END CERTIFICATE-----

$ openssl pkcs12 -nokeys -info -in  truststore-openssl.p12
MAC: sha256, Iteration 2048
MAC length: 32, salt length: 8
PKCS7 Encrypted data: PBES2, PBKDF2, AES-256-CBC, Iteration 2048, PRF hmacWithSHA256
Certificate bag
Bag Attributes: <No Attributes>
subject=C = CH, ST = Geneva, L = Geneva, O = SonarSource SA, CN = localhost
issuer=C = CH, ST = Geneva, L = Geneva, O = SonarSource SA, CN = SonarSource SA
-----BEGIN CERTIFICATE-----
[....]
-----END CERTIFICATE-----

we can see that the difference is in the Bag Attributes. The friendlyName and 2.16.840.1.113894.746875.1.1 attributes are missing.

I could not find a way with openssl to generate such attributes. I found this discussion and this issue, that might be solved in latest openssl, but I think most users will be stuck on older openssl version for a long time, and also this looks again very Java-specific.

Would you see a way for sslcontext-kickstart to accept openssl generated PKCS12 truststore out of the box?

[sureshg]

https://github.com/nhorman/openssl/commit/3e2b9aadbe553efa3094edeaffe0a77efed988b8

Use this option -jdktrust anyExtendedKeyUsage

$ openssl pkcs12 -export -jdktrust anyExtendedKeyUsage \
              -nokeys \
              -in server.pem \
              -password pass: \
              -out truststore-openssl.p12

[henryju]

Thanks @sureshg, I just found this new command line argument a few minutes ago. The issue is that it requires openssl 3.3.0+, and even the new Ubuntu LTS is still on 3.0.13, so I don't see this as a practical solution for our users at the moment.

Do you know by chance if those extra fields are useful from a security point of view, or are they only added by openssl/keytool to please the JDK? If the latter, I think this would be a cool addition to sslcontext-kickstart to circumvent this extra complexity (if possible).

[Hakky54]

I will investigate this the coming weekend. pkcs12 is a good format and it should just work out of the box. I am not familiar with the -jdktrust option for openssl. Let me check whether we can get it working without the need of that option.

4 years ago a had written https://github.com/Hakky54/mutual-tls-ssl which was mostly done in openssl. After some time I had rewritten the commands to keytool. In that time I didn't encountered this issue which you have.

I will check whether there is a different command needed for openssl or else it would be nice indeed to resolve this within the library itself.

[Hakky54]

As far as I could investigate/find out it seems like pkcs12 was initially just a container to hold private key pair entries. Oracle introduced the concept trusted entries in a keystore file. They initially had their own type aka JKS, this coud have private key pair and trusted certificate entries. I digged through the documentation of openssl and could not find a way to add for example a single Certificate Authority certificate. When I try to just add a ca certificate it gives me the error: Could not find private key from -in file from

And I think @sureshg hit the right spot of pointing to the option -jdktrust option as the documentation has the following explanation Mark certificate in PKCS#12 store as trusted for JDK compatibility Looking at the following openssl confirms my assumptions on this topic: https://github.com/openssl/openssl/issues/22215

It seems like adding that options provides some metadata or magic which ensures the certificate in the key pair is also added as a certificate entry next to the key pair entry.

In the java world with ssl the trustmanager only picks up the certificate entry skips the private key entry. The workaround is quite easy as extracting the certificate chain from the key pair and add that as a trusted certificate entry to the in memory keystore while loading it into the SSLFactory.

I think that would most likely work, however if I add that option for the existing flow it would also unintentionally impact other developers as it will automatically trust all the certificate chains of all key pair entries in a keystore, so in short it will trust additional certificates while maybe not desired by other developers.

If I would add a new method in the KeyStoreUtils for loading the keystore with certificates from the key pair, would that also work out for you?

[Hakky54]

@henryju

I created a PR which I think will resolve your issue. Can you maybe try to build a snapshot of the feature branch locally and try it out on your project/app? Looking forward whether it will actually work with the issue you have. You only need to add withTrustingCertificateChains to the SSLFactory builder. This option will scan the private key pair of the supplied keystores and get the certificate chain and it will use that as additional trusted certificates.

The PR is here: https://github.com/Hakky54/sslcontext-kickstart/pull/550 the branch is feature/trust-certificate-chain-from-private-key-pair

[henryju]

I digged through the documentation of openssl and could not find a way to add for example a single Certificate Authority certificate.

Do you mean adding a public certificate, without the corresponding private key? Isn't it the point of the -nokeys?

I will test your PR, thanks a lot!

[Hakky54]

Do you mean adding a public certificate, without the corresponding private key? Isn't it the point of the -nokeys?

I tried that, but that didn't work. If I use that option and just add a certificate it will create an empty keystore. But lets see what the outcome will be of your test with these changes

[henryju]

it will create an empty keystore

This is the command I am using to create the keystore:

$ openssl pkcs12 -export -name localhost -nokeys -in server.pem -out client-with-certificate-openssl.p12 --passout pass:pwdClientP12

Are you using keytool to "browse"?

$ keytool -v -list -keystore client-with-certificate-openssl.p12 
Entrez le mot de passe du fichier de clés :  pwdClientP12
Type de fichier de clés : PKCS12
Fournisseur de fichier de clés : SUN

Votre fichier de clés d'accès contient 0 entrées

keytool consider the keystore as empty, but using openssl, we can see it is not:

$  openssl pkcs12 -nokeys -info -in  client-with-certificate-openssl.p12 -passin pass:pwdClientP12
MAC: sha256, Iteration 2048
MAC length: 32, salt length: 8
PKCS7 Encrypted data: PBES2, PBKDF2, AES-256-CBC, Iteration 2048, PRF hmacWithSHA256
Certificate bag
Bag Attributes: <No Attributes>
subject=C = CH, ST = Geneva, L = Geneva, O = SonarSource SA, CN = localhost
issuer=C = CH, ST = Geneva, L = Geneva, O = SonarSource SA, CN = SonarSource SA
-----BEGIN CERTIFICATE-----
xxxx

[Hakky54]

I tried the keystores you have provided here: https://github.com/SonarSource/sonar-scanner-java-library/pull/195 And this library is not able to get the trusted certificates which is generated with openssl. The ones which are created with keytool and openssl with jdktrust option only works... I even tried with hooking up BouncyCastle with their custom security provider BouncyCastleJsseProvider and BouncyCastleProvider which have their own KeyStore and TrustManager implementations and unfortunatelly even those providers are not able to read it. It is just not able to read the certificates somehow.

It should be like this:

But it is:

So it seems like to fully migrate from keytool to openssl you need to have the -jdktrust option so the trustmanager is able to pick up the trusted certificate.

@sureshg do you maybe have any other ideas, maybe workarounds?

[henryju]

To solve this issue for our users (i.e., without requiring them to use keytool), I am considering supporting trusted certificates passed as PEM out-of-the box. It seems to be supported thanks to sslcontext-kickstart-for-pem.

So if this is possible to use a PEM as trust material, and assuming we can read a certificate from the pkcs12 keystore (or is it the problem?), would it be possible to do this: pkcs12 -> pem -> X509ExtendedTrustManager ?

[henryju]

I discovered something interesting. When creating a pkcs12 keystore with openssl without private key, the -name parameter is useless. I should have used -caname.

When using -caname the certificate had a proper alias in the keystore, and I could use Bouncycastle to load it:

        Security.addProvider(new BouncyCastleProvider());
        KeyStore keystore = KeyStore.getInstance("PKCS12", BouncyCastleProvider.PROVIDER_NAME);
        keystore.load(new FileInputStream(trustStore.getPath().toString()), trustStore.getKeyStorePassword().toCharArray());
        for (Iterator<String> it = keystore.aliases().asIterator(); it.hasNext();) {
          String alias = it.next();
          sslFactoryBuilder.withTrustMaterial(keystore.getCertificate(alias));
        }

So now, the question is: should I add Bouncycastle dependency to my project, or can sslcontext-kickstart support this kind of certificate out of the box?

[henryju]

I have updated https://github.com/SonarSource/sonar-scanner-java-library/pull/195 with openssl certificates generated using -caname, and using bouncycastle to load them.

[Hakky54]

Well discovered that -caname did the trick while using Bouncy castle!

I would rather not want to include bouncy castle as a transitive dependency in the core library as it will include 10 MB of transitive jars on the classpath. My goal of the core module is to keep it as light as possible with the bare minimum. So if you wanna go forword with using PKCS12 keystore including it on your side would be better.

We can simplify your example even further with the following snippet

KeyStore trustStore = KeyStoreUtils.loadKeyStore(Paths.get("/path/to/client-with-certificate-openssl.p12"), "pwdClientP12".toCharArray(), BouncyCastleProvider.PROVIDER_NAME);

SSLFactory sslFactory = SSLFactory.builder()
        .withTrustMaterial(trustStore)
        .build();

If you prefer to use pem files, then there is no issue at as the sslcontext-kickstart-for-pem module can handle all of those files.

So it seems like you can fully migrate from keytool to openssl. What would your preference be, using pkcs12 keystore file or pem file? And what is your opinion for adding the bouncy castle dependency on your side if you are planning to use the pkcs12 keystore files?

[henryju]

What would your preference be, using pkcs12 keystore file or pem file?

I am considering supporting both. I am now realizing that using pkcs12 as a truststore is a very Java-specific thing. Most other tools use pem as input.

And what is your opinion for adding the bouncy castle dependency on your side if you are planning to use the pkcs12 keystore files?

I think it's fine for us to use another third-party lib. I am also concerned by the size increase, but since we are only using bouncycastle for loading PKCS12, I wonder if I could not minimize the final JAR.

[henryju]

We can simplify your example even further with the following snippet

KeyStore trustStore = KeyStoreUtils.loadKeyStore(Paths.get("/path/to/client-with-certificate-openssl.p12"), "pwdClientP12".toCharArray(), BouncyCastleProvider.PROVIDER_NAME);

Are you sure this is correct? The third parameter of KeyStoreUtils.loadKeyStore is the keystore type, not the keystore provider.

We would need to add new overloaded methods in KeyStoreUtils, taking the provider as an additional parameter.

[Hakky54]

We can simplify your example even further with the following snippet

KeyStore trustStore = KeyStoreUtils.loadKeyStore(Paths.get("/path/to/client-with-certificate-openssl.p12"), "pwdClientP12".toCharArray(), BouncyCastleProvider.PROVIDER_NAME);

Are you sure this is correct? The third parameter of KeyStoreUtils.loadKeyStore is the keystore type, not the keystore provider.

We would need to add new overloaded methods in KeyStoreUtils, taking the provider as an additional parameter.

My bad, I pasted the wrong code snippet. Can you try:

KeyStore trustStore = KeyStoreUtils.loadKeyStore(Paths.get("/path/to/client-with-certificate-openssl.p12"), "pwdClientP12".toCharArray(), "PKCS12", BouncyCastleProvider.PROVIDER_NAME);

SSLFactory sslFactory = SSLFactory.builder()
        .withTrustMaterial(trustStore)
        .build();

It is available here so no new overloaded method is needed: https://github.com/Hakky54/sslcontext-kickstart/blob/823a951b4c6e63f2661d5f664783e7ff0e79f6c9/sslcontext-kickstart/src/main/java/nl/altindag/ssl/util/KeyStoreUtils.java#L99

[henryju]

I was still using 8.3.5, so I did not see the new method. It is indeed there in 8.3.6, thanks!

[Hakky54]

By the way, what did you guys decide on this topic. Will you be supporting PKCS12 and PEM files or just one option? It looks like both options work 😄

[henryju]

Our plan is to fix our support of pkcs12 by adding the Boucycastle library to our Java components (other ecosystems are not affected). Later, we might consider adding support for PEM, but this is a bigger change.

[Hakky54]

Sounds good, then I am closing this issue. If you later need help with the pem implementation feel free to drop a message here