Add support for PEM format SSL certificates

[Praveen-Rai]

Presently, Kestrel configuration supports only the PKCS12 ( *.pfx ) format certificates. Whereas, PEM seems to be much more widely used format ( Public and Private keys are stored in two different files ) and most often one needs to convert the certificate format using OpenSSL before they can be used with Kestrel.

Referring to : https://github.com/dotnet/corefx/issues/32367#issuecomment-423671246 It looks like that the core already has api's to load public and private keys from two different files. Can we leverage them and add the support for the PEM certificate formats ?

[Tratcher]

It looks like that the core already has api's to load public and private keys from two different files. Can we leverage them and add the support for the PEM certificate formats ?

Which APIs does core have for this? Do you end up with one or two X509Certificate2 instances?

Note Kestrel uses SslStream for all of it's TLS negotiation, we're constrained by what it can do.

[Drawaes]

SslStream doesn't care how you get your x509Certificate2 generally. As for the PEM issue not sure if you can construct the instance from two separate files but you can in theory (eg I modified core tondo it with openssl).

If it can't be done today it's most likely an ask for an API extension in .net corefx that is needed.

[Drawaes]

I will test it tonight because I had an example for Linux of this working.

[Praveen-Rai]

@Drawaes Any updates ??

[muratg]

@Praveen-Rai I recommend filing a bug on dotnet/corefx repo to track SSL format support ask. It's on their side to implement this.

[Praveen-Rai]

@muratg , @Tratcher I already created an issue earlier dotnet/corefx#32367 . The comments on the issue seems to convey that there's already some ground work done in Core and Kestrel needs to implement the changes to be able to support the PEM format. CopyWithPrivateKey() was cited in one of the comments. Request you to please review the issue once and correct me if i got it wrong and i will raise another issue specific to PEM format Support.

For your reference the API doc for CopyWithPrivateKey() :

• RSACertificateExtensions.CopyWithPrivateKey

• DSACertificateExtensions.CopyWithPrivateKey

[Tratcher]

Ah, since the X509Certificates2 APIs already exists then you can already load the certificates, combine them, and pass them to Kestrel.

What kind of additional Kestrel API do you anticipate is needed here? An overload of UseHttps that accepted two certificate paths? Additional config file parameters for specifying the second cert? Show us how you want to use it.

[Praveen-Rai]

@Tratcher Referring to Kestrel Listen Options

1. I couldn't find any Overload to UseHttps() which takes Public Key File (.crt) and Private Key File(.key).
2. Looking the sample configuration in the same article :

`

"HttpsInlineCertFile": {

  "Url": "https://localhost:5001",

  "Certificate": {

    "Path": "<path to .pfx file>",

    "Password": "<certificate password>"

  },

"HttpsInlineCertStore": {

  "Url": "https://localhost:5002",

  "Certificate": {

    "Subject": "<subject; required>",

    "Store": "<certificate store; defaults to My>",

    "Location": "<location; defaults to CurrentUser>",

    "AllowInvalid": "<true or false; defaults to false>"

  }

`

Quoting the statement below the sample :

The Certificate section supports both Path–Password and Subject–Store certificates.

All i am looking forward is something as example below :

`

"HttpsPEMCertificate": {

  "Url": "https://localhost:5002",

  "Certificate": {

    "PublicKeyFile": "<filepath to *.crt>",

    "PrivateKeyFile": "<filepath to *.key>",

  }

`

Presently most of the certificate providers issue a pair of .crt and .key file ( as far as my knowledge goes ). But as Kestrel configuration needs .pfx file, one needs to combine .crt & .key using OpenSSL to generate .pfx file.

[muratg]

@Tratcher do we need to add a sample/doc or are we missing a necessary API?

[Tratcher]

Both. Anything Kestrel can do with certs can also be done in app code (needs a sample), but Kestrel could also provide more first class APIs and config for this scenario.

[Praveen-Rai]

@Tratcher Any timelines for this issue ??

[Tratcher]

@Praveen-Rai no, we're just starting our 3.0 planning so that's the earliest release that could include something like this.

Putting together a sample of how devs would do this in Program.Main would be a good place to start. Kestrel's wouldn't be doing anything special here, only integrating and simplifying what devs can already do themselves.

https://github.com/dotnet/corefx/issues/32367#issuecomment-423671246 mentions the functionality being available in the corefx master branch. That also corresponds to the 3.0 release. That might be challenging to work with prior to the first preview release, you might need to revisit this issue after that.

[Praveen-Rai]

Ah, since the X509Certificates2 APIs already exists then you can already load the certificates, combine them, and pass them to Kestrel.

Putting together a sample of how devs would do this in Program.Main would be a good place to start.

@guardrex , @shirhatti Can we please have a document on how to read and load CRT format certificates using X509Certificates2 APIs and pass them to Kestrel ?

[Praveen-Rai]

@Tratcher Now that we have .Net Core 3.0 preview 1 released, can you please re-look into this issue ?

[Tratcher]

This is on our backlog, we don't plan to investigate it for 3.0. It's marked as up-for-grabs as a likely candidate for community involvement.

[Praveen-Rai]

@Tratcher So as an alternative we'll have to put up a sample in the Docs. Can you please help on the same ?

[Tratcher]

That would be almost the same work and is triaged with the same result. We'll review anything the community comes up with here but don't plan to investigate it ourselves right now.

[omerlh]

For future readers who might need something similar, this PR (Soluto/kamus#307) shows an example of loading a pem file for HTTPS handling in Kestrel. It was a bit ugly, but not that hard - once you collect all the different parts of information :) a documentation PR will be appreciated?

[ianpowell2017]

Any plans for this available in .net core 2.2 and below?

[Tratcher]

Any plans for this available in .net core 2.2 and below?

No, as a rule we do not backport new features or API. We haven't even scheduled this for a future release yet. That said, you should already be able to write app code for this in 3.0. If somebody works out a sample we'd be happy to add it to the docs.

[brzezinol]

I have a question about using pem as is. Today letsencrypt renew certificate for every 3 months. So I need another schedule for convert pem file to pfx file and copy to net.core app. Do yo know any other way to use let'sencrypt with net.core ?

[Brymastr]

@brzezinol My current workaround is to use openssl to convert the pem from letsencrypt to a pfx as a --deploy-hook in certbot. Example:

$ sudo certbot certonly \
    -d <DOMAIN NAME> \
    --deploy-hook cert-renew-hook

Where cert-renew-hook is something like:

#!/usr/bin/env bash
openssl pkcs12 \
  -inkey /path/to/privkey.pem \
  -in /path/to/cert.pem \
  -export -out /etc/letsencrypt/live/<DOMAIN NAME>/cert_pfx.pfx \
  -passout pass:

Then in my Kestrel config I reference the new pfx at the above directory.

This may be a bit naive as pfx requires a password but this example sets a blank string.

[rynowak]

@Tratcher - based on where we are now - what's the right fix for this look like? Is this something we'd want an API from CoreFx for? An API in ASP.NET? It seems like the approach that's obvious is a way to load a .pem as an X509Certificate.

[qJake]

Would really like to see support for this - since Let's Encrypt is quickly gaining traction, it is becoming increasingly common to have a fullchain.pem file that looks like this:

-----BEGIN CERTIFICATE-----
[Base64 of Cert]
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
[Base64 of Cert]
-----END CERTIFICATE-----

And a privkey.pem that looks like this:

-----BEGIN PRIVATE KEY-----
[Base64 of Private Key]
-----END PRIVATE KEY-----

Kestrel should support these formats natively, especially if it just means concatenating the files or something trivial like that.

[rynowak]

Some updates on the status here.

---

dotnet/runtime is tracking an issue for .NET 5.0 to expose a simple API for creating a cert from these file kinds - https://github.com/dotnet/runtime/issues/31944 Feedback is definitely encouraged from the community on whether that meets the goals.

This would allow you to pass in a certificate to the overloads of UseHttps that accept one. One possible problem is what to do with UseHttps(string), and whether it can support this format (probably, unless you have a separate key file). UseHttps(string, string) already exists and it does something different.

---

Additionally, we want to make it possible to pass in a certificate via configuration. Straw man proposal from @Tratcher

"Kestrel": {
    "Endpoints": {
      "HttpsPemCertFile": {
        "Url": "https://localhost:5002",
        "Certificate": {
          "Path": "<path to .pem file>",
          "PemKey": "<path to key file>"
         }
      },
}

Path would be able to accept a .pfx or a .pem. PemKey would specifically accept a private key - via file.

Personal note: I like the better the idea of KeyPath or something like that as a configuration key since that will reflect the fact that it's a file, not an inline value.

[Tratcher]

One possible problem is what to do with UseHttps(string)

If you're already doing code based config then you can call the new CreateFromPemFile API and pass in the cert. I'm not concerned about one extra line.

[rynowak]

One possible problem is what to do with UseHttps(string)

If you're already doing code based config then you can call the new CreateFromPemFile API and pass in the cert. I'm not concerned about one extra line.

Right, but we still have to think about how UseHttps(string) will behave because lots of folks will try that and expect it to do what they want.

[qJake]

I'd like to think people are smarter than passing actual certificate data into a parameter called fileName, but hey, maybe I give people too much credit. 😉

[rynowak]

I'd like to think people are smarter than passing actual certificate data into a parameter called fileName, but hey, maybe I give people too much credit. 😉

I meant for people passing the filePath of .pem certificate rather than a .pfx certificate.

[qJake]

Ah, hmm... would it be too difficult to auto-detect the file type and use it (given some list of known supported types), or create an overload that takes an enum that specifies the type (e.g. UseHttps("mycert.pem", X509CertificateType.Pem); or similar)?

[codyja]

Any ideas on when a solution would happen here? PEM cert reads would be ideal. Thank you!

[davidfowl]

Moving this out of backlog into 5.0.0

cc @anurse @shirhatti

[davidfowl]

Dependent on https://github.com/dotnet/runtime/issues/31944

[javiercn]

This is now merged into master and will be available in preview8