Serve bootstrap package from CloudFront

noahtalerman commented 1 month ago

Goal

User story
As a CPE with my Fleet hosted in a specific AWS region and who's configured Fleet to store the bootstrap package in S3,
I want to serve the bootstrap package from CloudFront
so that I can deliver the fastest download possible from the closest CloudFront region.

Context

Product designer: TODO

From the customer:

Bootstrap package is ~50MB.
We onboard new hires and replacement machines globally.
Our Fleet lives in a specific AWS region.
We very much benefit from storing our bootstrap package in S3 and serving the actual download from CloudFront. That way, machines get the fastest download possible from the closest CloudFront region, avoiding the long round trip to the AWS region where Fleet is hosted.

Proposed solution

3 new Fleet server config options:

cloud_front_url: "123456789.cloudfront.net"
cloud_front_key_id: "K123456789"
cloud_front_signing_key: "PEM body of RSA key"

1 CloudFront URL and AWS will handle routing the machine to the closest CloudFront edge location based on DNS and other regional data.

Changes

Product

[ ] Reference documentation changes: TODO
[ ] UI changes: TODO
[ ] CLI usage changes: TODO
[ ] REST API changes: TODO
[ ] Fleet's agent (fleetd) changes: TODO
[ ] Permissions changes: TODO
[ ] Changes to paid features or tiers: TODO

Engineering

[ ] Feature guide changes: TODO
[ ] Database schema migrations: TODO
[ ] Load testing: TODO

ℹ️ Please read this issue carefully and understand it. Pay special attention to UI wireframes, especially "dev notes".

QA

Risk assessment

Requires load testing: TODO
Risk level: Low / High TODO
Risk description: TODO

Manual testing steps

Step 1
Step 2
Step 3

Testing notes

Confirmation

[ ] Engineer (@____): Added comment to user story confirming successful completion of QA.
[ ] QA (@____): Added comment to user story confirming successful completion of QA.

noahtalerman commented 1 month ago

Thoughts on potential solution from the community:

Make sure the S3 prefix path that they store objects in is also set correctly on the CloudFront side, so that CloudFront can find the S3 object(s) to serve.

I'd suggest making each S3 object have some uniqueness in the name in order to avoid CloudFront catching weirdness.

Like, if an IT admin updates their bootstrap (or any) package and re-uploads it in place, you'd expect the CloudFront cache to get busted so that new clients download the newest version of the package.

dherder commented 3 weeks ago

@noahtalerman fo you want a separate issue to implement this same thing but for custom software packages?

noahtalerman commented 5 days ago

Luke: There's a delay (up to an hr) in which CloudFront won't be ready yet. Even if you invalidate the cache.

noahtalerman commented 5 days ago

Luke: Do we need the key_id and key?

Luke: Does the IT admin upload the S3, then wait for this to get added in CloudFront and then update the Fleet config? Or, does Fleet update the config for them?

Luke: Probably the latter.

zayhanlon commented 2 days ago

customer-starchik q4 requirement (can be evaluated today but possible for delivery any time in the quarter)

zayhanlon commented 2 days ago

@Patagonia121 i know this is a q4 key request for this customer but it wasn't selected for the current sprint due to capacity. we're focusing on a different ask from this customer. please bring it back to the next prioritization call

fleetdm / fleet