Programmatic keyfile creation and PEM file import

[openconcerto]

Hi,

Certificates providers are using the PEM format. Their documentation and the Jetty wiki is always refering a painful method using keytools and openssl.

Here is the better way you could include in the documentation, or better in the util package, following the code I wrote years ago (public domain :) ) and used in production.

    /**
     * Create a KeyStore from standard PEM file
     * 
     * @param privateKeyPem the private key PEM file
     * @param certificatePem the certificate(s) PEM file
     * @param the password to set to protect the private key
     */
    public static KeyStore createKeyStore(File privateKeyPem, File certificatePem, final String password)
            throws Exception, KeyStoreException, IOException, NoSuchAlgorithmException, CertificateException {
        final X509Certificate[] cert = createCertificates(certificatePem);
        final KeyStore keystore = KeyStore.getInstance("JKS");
        keystore.load(null);
        // Import private key
        final PrivateKey key = createPrivateKey(privateKeyPem);
        keystore.setKeyEntry(privateKeyPem.getName(), key, password.toCharArray(), cert);
        return keystore;
    }
    public static SSLServerSocketFactory createSSLFactory(File privateKeyPem, File certificatePem, String password) throws Exception {
        final SSLContext context = SSLContext.getInstance("TLS");
        final KeyStore keystore = createKeyStore(privateKeyPem, certificatePem, password);
        final KeyManagerFactory kmf = KeyManagerFactory.getInstance("SunX509");
        kmf.init(keystore, password.toCharArray());
        final KeyManager[] km = kmf.getKeyManagers();
        context.init(km, null, null);
        return context.getServerSocketFactory();
    }
    private static PrivateKey createPrivateKey(File privateKeyPem) throws Exception {
        final BufferedReader r = new BufferedReader(new FileReader(privateKeyPem));
        String s = r.readLine();
        if (s == null || !s.contains("BEGIN PRIVATE KEY")) {
            r.close();
            throw new IllegalArgumentException("No PRIVATE KEY found");
        }
        final StringBuffer b = new StringBuffer();
        s = "";
        while (s != null) {
            if (s.contains("END PRIVATE KEY")) {
                break;
            }
            b.append(s);
            s = r.readLine();
        }
        r.close();
        final String hexString = b.toString();
        final byte[] bytes = DatatypeConverter.parseBase64Binary(hexString);
        return generatePrivateKeyFromDER(bytes);
    }

    private static X509Certificate[] createCertificates(File certificatePem) throws Exception {
        final List<X509Certificate> result = new ArrayList<X509Certificate>();
        final BufferedReader r = new BufferedReader(new FileReader(certificatePem));
        String s = r.readLine();
        if (s == null || !s.contains("BEGIN CERTIFICATE")) {
            r.close();
            throw new IllegalArgumentException("No CERTIFICATE found");
        }
        StringBuffer b = new StringBuffer();
        while (s != null) {
            if (s.contains("END CERTIFICATE")) {
                String hexString = b.toString();
                final byte[] bytes = DatatypeConverter.parseBase64Binary(hexString);
                X509Certificate cert = generateCertificateFromDER(bytes);
                result.add(cert);
                b = new StringBuffer();
            } else {
                if (!s.startsWith("----")) {
                    b.append(s);
                }
            }
            s = r.readLine();
        }
        r.close();

        return result.toArray(new X509Certificate[result.size()]);
    }

    private static RSAPrivateKey generatePrivateKeyFromDER(byte[] keyBytes) throws InvalidKeySpecException, NoSuchAlgorithmException {
        final PKCS8EncodedKeySpec spec = new PKCS8EncodedKeySpec(keyBytes);
        final KeyFactory factory = KeyFactory.getInstance("RSA");
        return (RSAPrivateKey) factory.generatePrivate(spec);
    }

    private static X509Certificate generateCertificateFromDER(byte[] certBytes) throws CertificateException {
        final CertificateFactory factory = CertificateFactory.getInstance("X.509");
        return (X509Certificate) factory.generateCertificate(new ByteArrayInputStream(certBytes));
    }

Hoping it could help.

Best regards,

Guillaume

[joakime]

We can only accept code when submitted via a pull request with a git sign-off against a user that also has a filed Eclipse ECA. (The folks in Eclipse Legal are real sticklers to this, sorry)

See: /CONTRIBUTING.md

[openconcerto]

I know, but last time I did, it was a mess.

SO, do not copy-paste this code, be inspired to create your own :) It will take 30 minutes, no more.

Seeing everytime the boring process based on keytool & openssl is DEPRESSING.

[gregw]

I think it is worthwhile streamlining pem processing

[gregw]

Here is the relevant jetty documentation.

I see your code gives replacements for the command line utilities referenced in the documentation, but I'm not entirely sure how that is less depressing than the existing command line tools. That code would still need to be packaged so it could be run from the command line, and thus we'd just have different command line tools (and running java from the command line can be depressing in it's own right).

Am I missing something here?

[openconcerto]

Oh yes :)

Maybe am I biased because we always deploy Jetty in standalone, aka from our own main. We reduce deployment issues by not having to deal with error prone xml/ini configuration. Our apps are bundled in a jar with no dependencies, all is managed from code.

For https, we need one certificate (or more), a PEM file is the common form of certificate delivery by providers and by the game changer "let's encrypt".

So, if you can create dynamically your keystore from a PEM file with one line of code (or one line in an xml/ini file, if you preffer...) it's far easier and faster than dealing with command line tools. How many time/hack to automagically create the keystore with the keytool/cat/openssl commands?

Isn't this pseudo code simplier to understand than the method from Oracle ?

File pkey = new File("/etc/letsencrypt/live/mywebsite.com/privkey.pem");
File pcert = new File("/etc/letsencrypt/live/mywebsite.com/fullchain.pem");
KeyStore k = PEMImporter.createKeyStore(pkey, pcert, PASSWORD);

sslContextFactory.setKeyStore(k);

[gregw]

OK I'm going to have to ponder on this for a while... to work out exactly how we might use something like this. I can see that for some use-cases it would be very useful, but it is still difficult to make general purpose.

I'll edit the issue name to make it easier for others to find in the meantime...

[openconcerto]

Maybe just consider cases when you want to setup https without using a PEM certificate ;)

[boughtonp]

Isn't this pseudo code simplier to understand than the method from Oracle ?

File pkey = new File("/etc/letsencrypt/live/mywebsite.com/privkey.pem");
File pcert = new File("/etc/letsencrypt/live/mywebsite.com/fullchain.pem");
KeyStore k = PEMImporter.createKeyStore(pkey, pcert, PASSWORD);

sslContextFactory.setKeyStore(k);

Sure, that's simpler, but why not go a step further and have...

sslContextFactory.addLetsEncryptCerts();

...and it adds all the certs in the letsencrypt directory and does everything necessary.

It could have optional arguments for domain/path/key/cert in case anyone needs granular control, but I'd expect the overwhelming majority to leave everything as default.

It may well be useful to have more generic PEM functionality also, but I don't see any reason not to make supporting Let's Encrypt certificates as simple and hassle free as possible.

[openconcerto]

Hi,

Sure, for this purpose I use Acme4J : https://shredzone.org/maven/acme4j/index.html but I don't think that retreiving certificates is something that we want built-in a webserver.

Regards, Guillaume

Le mer. 5 juin 2019 à 01:02, Peter Boughton notifications@github.com a écrit :

Isn't this pseudo code simplier to understand than the method from Oracle https://docs.oracle.com/cd/E35976_01/server.740/es_admin/src/tadm_ssl_convert_pem_to_jks.html ?

File pkey = new File("/etc/letsencrypt/live/mywebsite.com/privkey.pem"); File pcert = new File("/etc/letsencrypt/live/mywebsite.com/fullchain.pem"); KeyStore k = PEMImporter.createKeyStore(pkey, pcert, PASSWORD);

sslContextFactory.setKeyStore(k);

Sure, that's simpler, but why not go a step further and have...

sslContextFactory.addLetsEncryptCerts();

...and it adds all the certs in the letsencrypt directory and does everything necessary.

It could have optional arguments for domain/path/key/cert in case anyone needs granular control, but I'd expect the overwhelming majority to leave everything as default.

It may well be useful to have more generic PEM functionality also, but I don't see any reason not to make supporting Let's Encrypt certificates as simple and hassle free as possible.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/eclipse/jetty.project/issues/1826?email_source=notifications&email_token=AAJNKV457OCY3QMUFKS2MCDPY3X65A5CNFSM4D3HRAP2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODW6DOYI#issuecomment-498874209, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJNKV5I6XV7INUZVY4TLALPY3X65ANCNFSM4D3HRAPQ .

[boughtonp]

I'm not asking for Jetty to do certificate management.

Your proposal is to avoid the horrible process of converting PEM certificates to a keystore - which I fully agree with - but I'm suggesting it should go further and also have a single simple switch to tell Jetty "I use Let's Encrypt with default settings, go ahead and connect my keys and certificates".

[stale[bot]]

This issue has been automatically marked as stale because it has been a full year without activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been closed due to it having no activity.

[sbordet]

I think we need to give this one more attempt.

LetsEncrypt is now widespread, and we use it for our own website, where we do have a script that runs on cron when the certificates are renewed and that converts the PEM certificates (LetsEncrypt) to PKCS12 (Java).

The problem is that in Java everything is based on the concept of a KeyStore. We modeled SslContextFactory around that, so we have SslContextFactory.keyStorePath which cannot be used to point to the two paths that LetsEncrypt uses to identify the cryptographic material (the private key and the full certificate chain).

While it'll be easy to write an importer utility class, we also want to be able to do this using properties for a standalone Jetty.

@gregw how about this:

startd/ssl.ini

jetty.sslContext.keyStorePath=PEM:/path/to/private.pem|/path/to/certs.pem

We do the conversion internally from the 2 paths in PEM format to a PKCS12 keyStore and we should be good even with reload().

Thoughts?

[openconcerto]

Sure, it's the way to go.

I would add that it could be interesting to add a setting to specify the delay to consider that the pem files should be reloaded.

In the case of of letsencrypt, the certificates are provided as symlinks pointing to the last "version" of the files (updated with a cron). So it could be good, to "refresh" the SslContext without having to restart Jetty.

In production, at start the webserver uses the PEM importer I shared ; and our standalone webserver is restarted every day with cron. Auto reloading of the SslContext could be great to avoid the 3 seconds downtime.

[knaccc]

I use jetty to host two different domains, requiring two different certificates from LetsEncrypt.

This jetty.sslContext.keyStorePath=PEM:/path/to/private.pem|/path/to/certs.pem approach would therefore not be suitable for situations where the keystore needs to hold more than one certificate, unless you allowed pairs of private.pem/certs.pem to be specified.

In my opinion, the above approach isn't as simple for the user as it could be.

A much simpler approach would be to have a certbot module for Jetty, that would monitor the /etc/letsencrypt/live folder for changes, and automatically create a keystore which it would then hot-reload into Jetty. If the /etc/letsencrypt/live folder contains multiple certificates, the keystore would contain multiple certificates unless manually configured otherwise.

This would mean that the only configuration a user would need to do is to add the Jetty certbot module, and everything would be automagically configured and updated when certbot renews the certificates.

[sbordet]

@knaccc good points.

[sbordet]

However, /etc/letsencrypt/live contains the private keys and as such typically owned by super-users that are not used to run Jetty.

[knaccc]

However, /etc/letsencrypt/live contains the private keys and as such typically owned by super-users that are not used to run Jetty.

Oops, yes, well caught :) It's pretty easy to have a 1-line bash script as a --post-hook on certbot to use openssl to convert the pem files to a pkcs12 keystore file and place it inside jetty, after which a future hot-reload module would detect the change and perform the reload. I'd suggest the main reason people struggle is that it's difficult to google a tutorial on how to do this.

So I guess the Java code for doing the equivalent of what openssl would otherwise do is really only required for people without the ability to run openssl.

[boughtonp]

Permissions are a common problem with known common solutions which apply equally to all servers/software.

So the documentation for the new module(s) can include an explanation with an example snippet such as:

# run once only
groupadd certusers
usermod -aG certusers jettyuser

# run on first setup of new certificate
cd /etc/letsencrypt/live/$domain
chgrp certusers ./privkey.pem
chmod g+r ./privkey.pem

Something similar would be needed for both a generic PEM module which was manually configured, as well as for an automatic letsencrypt module that scanned the directory.

The latter would also require running chmod 0755 /etc/letsencrypt/{live,archive} (or simply chmod +rx /etc/letsencrypt/{live,archive}) to allow it to list the directory contents, (which is not needed if accessing a specific domain directory as the manually configured module would do).

[gregw]

@lachlan-roberts what is the status of this one?

[sbordet]

@gregw @lachlan-roberts I'm not sure we can do much here.

The idea to support two PEM files instead of 1 KeyStore file won't work easily due to file permissions. Having a Java tool that does PEM to PKCS12 conversion it's the same as using OpenSSL -- it would need to be hooked to LetsEncrypt certificate renewals.

As there is a million ways to skin and setup files, permissions, hooks, etc. I'm not convinced there is much we can do here -- but I can be convinced otherwise.

[openconcerto]

What do you mean by "there is a million ways to skip and setup files, permissions, hooks" ? Is throwing an exception if the pem files are not readable a problem? The java.io.File API provide all you need to throw a meaningful cause.

Nginx and Apache provide a simple (builtin) way to use pem files. I don't understand why on Jetty it should not be as easy.

I'm still quite perplexed about assertion like "it can be done by external tools". (Sure, but I could argue that SSL support is too, so let's remove https from Jetty because haproxy can replace it ;) )

[sbordet]

It's not a problem throwing an exception if you can't read a file. The problem is always (or almost always) throwing it because the files are owned by root, and the Jetty user (almost always) does not have permission to read them: what's the point of writing code that always fail?

Nginx and Apache link to the OpenSSL library. Java does not link it, so Jetty does not have a mean immediately available to read PEM files. Sure we can do it, but again see the comment above.

My point is that ok, we can write in Java the tool to convert from PEM to KeyStore. But then, how differently will this tool be used from openssl? If there is no difference, why should we write a tool in the first place?

Let's take the LetsEncrypt case: upon renew by certbot, we need to convert the PEMs to KeyStore. We do this using a script in /etc/letsencrypt/renewal-hooks/post/, run by the user that runs the cron job that runs certbot (i.e. root). We use openssl to do the conversion.

What do you suggest?

[openconcerto]

because the files are owned by root, and the Jetty user (almost always) does not have permission to read

It's the same with Apache & Nginx, file permissions are granted as needed (through file permission or a group).

Nginx and Apache link to the OpenSSL library. Java does not link it

Sure, but Java can read PEM using X509Certificate as shown in my first message. Or did I miss something?

In the case of a standard or EV certificat renewal, ie a certificat you renew and copy every year (or 2...) on your server, it's more "user friendly" to just copy the new PEM without having to remember the openssl and the keytool error prone syntax.

In the case of cerbot, it all depends of the logic you use/need. IMHO it's not a the best to rely on a external system (cron) to decide when to restart the webserver. The best being not to restart the webserver but only refresh the keystore. In our use cases, caches/whitelist/blacklist/tokens are stored in memory for performance and security reasons, so a restart is not great...

[sbordet]

I am missing your point, sorry.

it's more "user friendly" to just copy the new PEM without having to remember the openssl and the keytool error prone syntax.

You have to remember what you copy and where, point being you have to remember something and that is error prone.

The ideal solution would be to automate everything.

That's what the cron job does: invokes certbot twice a day, and if the certificate is renewed, it runs scripts stored in a well-known location, where you have written the stuff that you need to do, using openssl to convert the PEMs, generating a KeyStore, replacing (carefully) the KeyStore, updating permissions, and reloading Jetty's SslContextFactory via e.g. JMX -- no server restart if you don't want to.

All of that is external to Jetty anyway.

I don't think implementing a PEM-to-KeyStore tool in Java is any better than openssl, given all the rest that you still need to have and setup externally to Jetty.

What do you think we should change in Jetty? I don't see anything evident given the use case above.

[lachlan-roberts]

I think we can close this one.

We now have the ssl-reload module which will hot reload the keystore if any changes to the file are detected. This can be used with a certbot post-hook script to generate the keystore file and switch out with the one used by Jetty which will be automatically detected and switched without needing to restart the server.

[openconcerto]

Humm... so you are stating that a post-hook has to create a keystore?

Why not, but it will expose the keystore password in the post-hook script.

Ouch.

[sbordet]

Why not, but it will expose the keystore password in the post-hook script.

That is exposed anyway because it's necessary to create a KeyStore. And it is in a file owned by a super-user.

Your example code in the first comment will have to be invoked with an exposed password, or it should generate a password that needs to be exposed to Jetty in ssl.ini.

I don't see a way around?

[openconcerto]

Sure, but it's a bit more "hidden" in a jar file if its generated.

I will look at how the "ssl-reload" works to use it to monitor the files and reload the keystore from PEMs.

But I'm still a bit perplexed to see how a standard thing like loading a pem file (as other webserver do) with a so short code addition, is not included (a bit like JSON in the JRE...).

[sbordet]

But I'm still a bit perplexed to see how a standard thing like loading a pem file (as other webserver do) with a so short code addition, is not included (a bit like JSON in the JRE...).

Because:

• it's not standard in Java (neither to load a PEM, nor to run the Java process with enough permissions to read the private key PEM);
• the code is not so short when you want to make it generic;
• it's way more complicated when you want to support multiple domains with different private keys;
• you still need an external post-hook script to reload the Jetty configuration when the PEMs are updated.

On the other hand:

• you need a post-hook anyway, so add in there 1 more line (literally) to convert PEMs to KeyStore via openssl; done.

[Hakky54]

I noticed this has been closed, but I think it is still possible to have this kind of option built in into Jetty itself. When I use jetty with pem files I use my own library to easily configure the ssl configuration, maybe this can be integrated. What do you guys think?

I am using the following snippet:

import nl.altindag.ssl.SSLFactory;
import nl.altindag.ssl.jetty.util.JettySslUtils;
import nl.altindag.ssl.pem.util.PemUtils;
import org.eclipse.jetty.util.ssl.SslContextFactory;

import javax.net.ssl.X509ExtendedKeyManager;
import javax.net.ssl.X509ExtendedTrustManager;
import java.nio.file.Paths;

public class App {

    public static void main(String[] args) {
        X509ExtendedKeyManager keyManager = PemUtils.loadIdentityMaterial(Paths.get("/path/to/your/chain.pem"), Paths.get("/path/to/your/private-key.pem"));
        X509ExtendedTrustManager trustManager = PemUtils.loadTrustMaterial(Paths.get("/path/to/your/some-trusted-certificate.pem"));

        SSLFactory sslFactory = SSLFactory.builder()
                .withIdentityMaterial(keyManager)
                .withTrustMaterial(trustManager)
                .build();

        SslContextFactory.Server sslContextFactory = JettySslUtils.forServer(sslFactory);
    }
}

See here for the library itself: sslcontext-kickstart It parses the pem files and creates an inmemory keystore object which will be used to construct the keymanager/trustmanager. So no need for a post-hook with openssl.