Certificate Lifecycle Management for Linux

[ddribeiro]

Problem

As an IT admin, I'd like Fleet to orchestrate the lifecycle management of client certificates on my Linux hosts.

There are several parts to this request that might need to be broken down into smaller stories:

1. Create a system to generate and issue client certificates
2. Automatically renew client certificates when they are about to expire
3. Have the ability to revoke client certificates through Fleet
4. Prevent a device that has had its certificate revoked from generating a new one if it gets re-imaged (persist device record in Fleet based on UUID or other identifier)

   What have you tried?

customer-cisneros is using scripts and Ubuntu Landscape to achieve this today.

1. A script is used to generate a private key on device
2. A script is used to generate a CSR on device
3. A script is used to pull the CSR from the device and submit it to a PKI service
4. Certificate is generated on the server and deployed to client device using Landscape

This workflow is able to renew certificates before they expire. It does not handle revocation.

Potential solutions

The solution for the customer would be to build a system in Fleet that replaces their current workflow and meets the requirements in the above sections. I don't have any solutions on how to best achieve this.

What is the expected workflow as a result of your proposal?

The expected workflow is that a Fleet admin would be able to use Fleet to manage client certificates on their Linux hosts instead of needed to build a custom workflow to handle this.

[JoStableford]

Related to a Slack conversation

[noahtalerman]

Thanks for tracking this @ddribeiro.

I think the plan is to use fleetdm.com (as a server that checks for expired certs) and script execution features in Fleet for now.

At some point in the future, when we have additional engineering capacity, we'll add this to Fleet.

cc @zwass @zayhanlon

[noahtalerman]

I think the plan is to use fleetdm.com (as a server that checks for expired certs) and script execution features in Fleet for now.

Confirmed w/ @zayhanlon. Removing from feature fest for now.

cc @ddribeiro

[phtardif1]

@pboushy - FYI

[dherder]

@noahtalerman Here's one way to do this with certmonger and script execution. https://docs.portnox.com/topics/onboarding_uem_linux_scep. The solution involves local private key generation on the linux endpoint, which might be a blocker for some organizations that would prefer the Fleet server to do the brokerage of the CSR generation and renewal / revocation workflow.

[dherder]

see #22452 @noahtalerman

[noahtalerman]

Here's one way to do this with certmonger and script execution. https://docs.portnox.com/topics/onboarding_uem_linux_scep. The solution involves local private key generation on the linux endpoint, which might be a blocker for some organizations that would prefer the Fleet server to do the brokerage of the CSR generation and renewal / revocation workflow.

Thanks @dherder :)

FYI @zwass and @zayhanlon re certificates on Linux

[dherder]

@noahtalerman I think the major challenge here is to ensure that the fleetd has an understanding of the certificate revocation process and renewal. Just with script execution, I can't see how to do this with just script execution. The fleetd has to have some logic to be able to track issuance, revocation, and renewals.

[noahtalerman]

major challenge here is to ensure that the fleetd has an understanding of the certificate revocation process and renewal. Just with script execution, I can't see how to do this with just script execution.

@dherder I think customer-cisneros's plan to address this is to have a another server/third-party tool to handle tracking this. That tool hits the Fleet API to run scripts.

I could be wrong.

cc @zwass

[pboushy]

Thanks for the mention @phtardif1.

This is an area I've spent a lot of time researching both in Mac and Linux.... and there doesn't seem to be a good solution that works across multiple Linux distros.

For me, the most critical items are:

1. Only devices that are enrolled in Fleet can request and retrieve a certificate. (this appears to be the most complex piece to script a solution because all the standards use static passphrases)
2. No static passphrase used to request the certificate. Use some form of authentication that is unique per client device.
3. The option to generate the private key and CSR locally on the device.
4. Ideally the private key should be locked to the device that requested it. In Windows or Mac, you do this by storing it in TPM or SecureEnclave respectively.
5. Monitor the expiration date of the certificate and kickoff a renewal process x% of the lifespan (e.g. renew 10% of the lifespan = 36/365days; 10/100days. This prevents the need to adjust as cert lifespan shrinks) - you can use a command like this: openssl pkcs12 -in "${cert}" -passin "${passin}" -nokeys -clcerts | openssl x509 -noout -enddate | cut -d = -f 2

For actually getting the certificate from a PKI environment, there are several options:

• Fleet proxies the request for a certificate and either:
  1. uses a certiticate request standard to retrieve the cert (e.g. SCEP, ACME), or
  2. integrate with multiple PKI services (Symantec, Sectigo, DigiCert, AD CS, Venafi)
• Configure the client to use SCEP or ACME - the biggest problem with this is that it usually fails critical item 1.

For certificate revocation, I believe that both SCEP and ACME support revocation through the APIs. If you instead were to integrate with multiple PKI services individual configurations, you would need to integrate the revocation for each individually.

[nonpunctual]

@noahtalerman I think the major challenge here is to ensure that the fleetd has an understanding of the certificate revocation process and renewal. Just with script execution, I can't see how to do this with just script execution. The fleetd has to have some logic to be able to track issuance, revocation, and renewals.

cc @dherder I could be very wrong but it kind of seems like we already have the mechanism for watching attributes of any cert on any device, ie, the certificates osquery table which is cross-platform...

The new part would be that in the Fleet db there could be a table that stored expiry for "Fleet-issued" certs & when the expiry reached some safe period prior to expiration like 14d, a command could be issued to renew.