Deploy process

[venkatd]

We need to have a consistent deploy process so that things get deployed to production with more reliability.

We can have a deploy checklist for production

• Extra items on the list if there is a migration (backup database, etc)
• Extra items on the checklist if the migration is long running

We can have insight into the status of different features

• What PRs are on staging/production? Which ones aren't?
• What PRs have been tested by a human on staging? Was the latest commit tested?
• What PRs have been tested by a human on production? Was the latest commit tested?

We can have a public schedule of when our deploys are happening. For example, the clients are expecting that we will deploy Reissued to production Monday 3/15 at 6:00 AM PST. This information isn't readily known to everyone on the team.

We can have a consistent weekly time to deploy to production. Ideally it's not 6 AM PST and at a time when more of us are at hand in case problems arise.

In addition to an integration test, we can have a checklist for a human testing the feature. This checklist should be written by the person building the feature. So the process would be as follows:

• Deploy to production via a PR from development to master
• Once the PR is deployed, run through the testing checklist for each feature to make sure each feature works as expected
• If a feature doesn't work as expected, create Asana tasks for them and assign them to whoever is up.
• We create hotfixes using code hotfix

People should be expected to be actively available during the deploy time

[joshsmith]

We should create a recurring task for 6AM the next morning and ensure it's assigned to someone by EOD Friday.

[venkatd]

@JoshSmith updated the description with several more points. I agreed to 6 AM tomorrow but I think it is safer to deploy at a time when most of us are available to fix problems that arise.

[venkatd]

This is RFC is now generally related to our whole deploy process

[venkatd]

@JoshSmith what is a good time to deploy normally? Before standup on Monday? After standup on Monday?

[joshsmith]

6:00AM PT would be best. Both you and Israel are available for overlap during that time.

The problem is also not deploying while people are actively using the site right now, until we have rolling deploys.

[venkatd]

6 AM - San Diego - sleep time 8 AM - Columbia - Israel should be able to be available by this time 9 AM - New York - I can be up 2 PM - Croatia - reasonable for Nikola to be available

What are rolling deploys?

[idelahoz]

+1 to 6:00 PT.

[joshsmith]

By rolling deploys I meant a deploy that's zero downtime.

[joshsmith]

It sounds like we're rallying around a 6 AM PT deploy time?

[begedin]

There's literally no reason for me not to be available at that time, so yeah, my vote is also 6:00 PT. Anything withing 1-2 hours before and after that time is also fine.

[venkatd]

Closing.