Add fleetd update channels: 1. Enable scripts 2. Renew certificates

[alexmitchelliii]

Goal

User story
As a Fleet user that deployed fleetd w/ autoupdates turned on,
I want there to be a fleetd update channel to enable scripts or renew certificates
so that I can enable scripts or renew certificates w/o asking my end users or clients to install a new fleetd agent.

Context

• Requestor(s): @alexmitchelliii

• Product designer: @noahtalerman

• Today, prospect-montague has deployed fleetd to Linux workstations. They want to enable scripts w/o deploying a new fleetd agent. Scripts cannot be enabled remotely (intentional feature for read-only customers).

• Today, a Fleet Free user has deployed fleetd to tens of thousands of hosts. They need to update the certificate bundle for their fleetd. The certificate bundle cannot be renewed remotely (Fleet hasn't gotten to building this yet).

Changes

Product

• [ ] Fleet's agent (fleetd) changes:
  • @zwass: Develop two custom agent builds: One includes code that enables script execution and the other renews certificates.
  • @lucasmrod or @roperzh: Push the builds with enable-scripts and renew-certificates tags respectively to the Fleet-hosted public-facing update server.
  • @lucasmrod and @roperzh: Provide support while the prospect use the update channel configuration in Fleet to remotely tell the agents to update to that version.
  • When agents update, the custom code will write a configuration change that enables script execution or renews certificates respectively.
  • @lucasmrod and @roperzh: Provide support after leaving the update channel set to the build for 30 days to catch as many agents as possible, and let the prospect and user know that they can return to using the regular “stable” update channel.
  • @lucasmrod or @roperzh: After both the prospect and the user have updated all agents, remove the builds with enable-scripts and renew-certificates tags respectively from the Fleet-hosted public-facing update server.
• [ ] UI changes: None.
• [ ] CLI usage changes: None.
• [ ] REST API changes: None.
• [ ] Permissions changes: None.
• [ ] Changes to paid features or tiers: Available in Fleet Free and Fleet Premium

Engineering

• [ ] Reference documentation changes: TODO
• [ ] Usage documentation changes: TODO
• [ ] Database schema migrations: TODO
• [ ] Load testing: TODO

ℹ️  Please read this issue carefully and understand it. Pay special attention to UI wireframes, especially "dev notes".

QA

Risk assessment

• Requires load testing: TODO
• Risk level: Low / High TODO
• Risk description: TODO

Manual testing steps

1. Step 1
2. Step 2
3. Step 3

Testing notes

Confirmation

1. [ ] Engineer (@____): Added comment to user story confirming successful completion of QA.
2. [ ] QA (@____): Added comment to user story confirming successful completion of QA.

[noahtalerman]

Updated this issue to user story format and moved the original issue description here:

Problem

Goal: Silently, and remotely enable script execution for already deployed agents (note that this is intentionally not possible generally to preserve read-only guarantees for security-sensitive users)

Context: Company doesn't have any management tool to push a new agent to the devices (Fleet script execution would be that management tool for them moving forward if accomplished). It's a bootstrapping problem.

Potential solutions

1. Develop a custom agent build that includes code that enables script execution when it detects it is running within prospect’s Linux deployment (detected by the enroll secret used).
2. Fleet will push that build with a custom tag on the Fleet-hosted public-facing update server.
3. Prospect can then use the update channel configuration in Fleet (https://fleetdm.com/docs/configuration/agent-configuration#configure-fleetd-update-channels) to remotely tell the agents to update to that version.
4. When agents update, the custom code will write a configuration change that enables script execution.
5. After leaving that setting for 30 days to catch as many agents as possible, prospect can return to using the regular “stable” update channel.

[noahtalerman]

Hey @zwass heads up, I updated the issue to user story format. When you get the chance, can you please take a look at the fleetd changes section? That's my understanding of what needs to happen. Please feel free to tweak it.

After we get a signed order form from prospect-montague please feel free to assign the issue to yourself.

Plan is in the doc here:

Heads up @roperzh and @lucasmrod, I think we'll need your help once Zach writes the code (see the "Changes" section in the issue). At that point, @zwass please add the issue to either the MDM or Endpoint ops sprint board and assign @georgekarrv (MDM EM) or @sharon-fdm (Endpoint ops EM).

cc @alexmitchelliii @lukeheath @AnthonySnyder8

[noahtalerman]

Fleet Free user: All binaries are unique because we have user binaries built in to them.

Zach: Some things changed in fleetctl package but not in fleetd binary. Changes would be taking place in fleetd binary.