[Tooling] Move site creation into CMS backend

Description

We initially started working on a site-creation feature using a lambda-based architecture - thanks @rc-davis for working on this first version of the feature.

After some discussion with the rest of the engineering team, we realized that Isomer could benefit from having the feature live in the main CMS backend codebase. The main benefit of this decision is that we can perform database operations more frequently and easily. As such, we are able to better ensure data integrity because we can more easily associate the user record to the site record.

Breakdown of issue

This issue can be broken down into 5 main PRs:

Create and update database tables and add the relevant migrations (https://github.com/isomerpages/isomercms-backend/pull/451)
Add a new ReposService and DeploymentsService with methods to create a new GitHub repo and AWS Amplify project https://github.com/isomerpages/isomercms-backend/pull/456
FormSG authentication middleware https://github.com/isomerpages/isomercms-backend/pull/458
Refactor MailClient to add a method to send emails on site creation https://github.com/isomerpages/isomercms-backend/pull/457
Add a new POST /sites route + createSite() method to SitesService to tie the abovementioned PRs together

Sequence diagram

The following Plant UML diagram was created on planttext.

Sequence description

The user submits a FormSG form with the necessary information to create a new site. The form currently has the following fields: _governmentemail, agency, _repositoryname, _sitename, _point_ofcontact.
FormSG sends a webhook payload to the POST /sites endpoint on the CMS backend API server. The API server does the following: a. It authenticates the FormSG request. The endpoint returns 200 if the request is valid; it returns 400 if the request is not valid. b. It checks the Users table to find a user record with an email field that matches the _governmentemail field in the form. c. If no such user is found, the API server sends an email to the user (obtained from the _governmentemail field) with the error message.
The API server then creates a new site record in the Sites table. The site record's name field will be set to the _sitename in the FormSG form submission data. The site record will also have INIT and RUNNING as their default values for the _sitestatus and _jobstatus fields. The site record will also be associated with the user obtained in step 2b via the creator field.
The API server creates a new GitHub repository with the name _repositoryname. It also creates a staging and master branch on the repository.
If the GitHub repo creation step fails: a. The API server updates the site record's _jobstatus to FAILURE. b. The API server then emails the user with the error message.
The API server creates a new repo record in the Repos table, and associates it with the existing site record in the Sites table.
The API server then creates a new AWS Amplify project with staging and master deployments.
If the Amplify project creation step fails: a. The API server updates the site record's _jobstatus to FAILURE. b. The API server then emails the user with the error message.
The API server creates a new deployment record in the Deployments table, and associates it with the existing site record in the Sites table.
The API server updates the site record's _sitestatus to LAUNCH _jobstatus to READY.
The API server then emails the user with the success message.

Plant UML code


@startuml
actor "Agency User" as user
box "Isomer CMS (AWS)"
participant "CMS backend (EB)" as eb
database "CMS database (RDS)" as db
end box
participant "GitHub" as gh
participant "Amplify" as amp

group Site creation
  group Init site creation
    user -> eb : 1. User submits Form
    eb -> eb : 2a. Authenticate form submission
    eb -> db: 2b. Check if user exists in the Users table
    eb -> user : 2c. If invalid, email user with failure message: EXIT
    else Else, proceed to 3
    eb -> db: 3. Create site record in Sites table
  end
  |||
  |||
  group Create GitHub repo
    eb -> gh: 4. Create GitHub repo with staging and master branches
    eb -> db: 5a. If 4 fails, Update site record, then
    eb -> user: 5b. Email user with failure message: EXIT
    else Else, proceed to 6
    eb -> db: 6. Create repo record in Repos table
  end
  |||
  group Create Amplify project
    eb -> amp: 7. Create Amplify project with staging and master deployments
    eb -> db: 8a. If 7 fails, Update site record
    eb -> user: 8b. Email user with failure message: EXIT
    else Else, proceed to 9
    eb -> db: 9. Create deployment record in Deployments table
  end
  |||
  group Complete site creation
    eb -> db: 10. Update site record
    eb -> user: 11. Email user with success message: EXIT
  end
end

@enduml

isomerpages / isomercms-backend