artsy / artsy.github.io

The Artsy Engineering Open-Source Developers Blog
https://artsy.github.io
MIT License
1.09k stars 227 forks source link

Comments: The Checklist for Deploying a Scary Change #754

Open mdole opened 1 year ago

mdole commented 1 year ago

Lately, I’ve been getting involved with some sketchy stuff. You know what I’m talking about–data migrations.

I’ve been rolling out changes that have a significant risk of breaking our production environment for mission-critical services. It’s been exciting work (keep your eyes out for more posts on the exact project, coming soon™️), but I’ve definitely caused a couple incidents along the way.

After accidentally taking down a key service for a couple hours, I decided I needed to have a better pre-deploy process for these changes. I did some thinking and came up with a short checklist to run through before I press the shiny green button.