python / pythondotorg

Source code for python.org
https://www.python.org
Apache License 2.0
1.49k stars 592 forks source link

Manage Fastly service configuration via Terraform #2510

Closed ewdurbin closed 1 day ago

ewdurbin commented 3 weeks ago

Describe the solution you'd like

Our current Fastly configuration is managed manually. This isn't ideal.

PSF Infrastructure currently manages PyPI's suite of Fastly services with Terraform via https://github.com/pypi/infra, and peps.python.org via https://github.com/python/peps/tree/main/infra. I'd like to see us do the same for www.python.org.

We have options, either a new Fastly service built from scratch to match the current service, or a Terraform import of the existing service (to be clear I don't think that constructs a terraform config, just imports the state).

This will enable us to have better safety and visibility in changes to that portion of the infrastructure.

humitos commented 3 weeks ago

Hi 👋🏼 -- I'm not really related to this project, but I'm taking the liberty to comment here with my experience on Terraform + Cloudflare for Read the Docs since I had to do similar work there and I think it could save you some time. You can just ignore it, tho 😄

or a Terraform import of the existing service (to be clear I don't think that constructs a terraform config, just imports the state).

Cloudflare has a cf-terraforming tool that helps you importing these configs. I suppose that Fastly could have something similar. It helped me with 50-70% of the job here. It's not perfect, but it saved me a good amount of time.

Apart from that, for the rest of the work that I had to do manually, I followed these steps for any resource:

  1. declare the resource I want to import almost empty, by only defining the required attributes
  2. use terraform import <resource> to import the state
  3. run terraform plan -target <resource> to see the diff between the empty resource and the one that's in production
  4. copy and paste that diff output into the resource code and adapt it from there
  5. run terraform plan -target <resource> until there are no diff anymore 😄

Depending on the resource itself, it could be a pretty tedious work or super simple. Good luck 😸

JacobCoffee commented 1 day ago

This was helpful, @humitos - thanks! :)

Completed in https://github.com/python/pythondotorg/pull/2519