Improve QA App Deployment & Performance

[sf-dcp]

Issue Description

We would like to improve deployment and maintenance process of the QA app. We currently use DO Droplet service (comparable to AWS ec2 instance) to host our QA app. Some issues that have come up while using Droplet are:

• Droplet infra maintenance
• Having to redeploy the app via Github Actions every time we make code changes
• Silent app failures. We would find out that something went wrong when checking the QA website

Proposed Approach

Migrate the QA app from Droplet to the DO App Platform service. App Platform is a Platform-as-a-Service and it takes care of the underlying infrastructure. There are multiple options to deploy an app to App Platform; in our case, the most convenient option seems to deploy app from a container image. This link describes the process.

Components of the approach:

• Deploy app. 2 Options:
  • Manually - use control panel in DO to configure the app
  • Programmatically - use DO CLI command doctl apps create. Configurations should be passed via an input yaml file app-spec.yml in DO. This file should include details such as app name, image name, build/runtime variables, and an App Platform billing plan.
• Manage app (i.e. redeploy). Again, can do it both manually or programmatically. See the list of all CLI commands to work with an app.
  • For the current process to redeploy the app, we can modify the existing workflow file to 1) update image and 2) call CLI command to re-deploy the app using existing configs.

Considerations of Proposed Approach

Deployment

With the proposed approach, we still need to redeploy the app manually if we choose to use an image directly from Docker hub (from Github Actions). App Platform offers automatic re-deployment of source code coming from DO registry images or a Github repo.

Failures

[TODO]

Cost

For the platform app basic plan with the specs similar to the current plan (except hard disk), the cost is ~$20/month. I believe this pricing is for 1 node and we have to choose a pro plan to have multiple nodes. We are currently paying ~$16/month (see Damon's comment below).

Security

[TODO]

Other

App platform is not suitable for storage. For example, we may have logic where we pull data from s3 to droplet for a qa page (ex: data viz). This won't work with the app platform; instead, we would have to pull data directly from DO storage OR add a volume resource (extra $$).

Relevant references:

• Create app on App Platform programmatically
• Pros and Cons of App Platform

[damonmcc]

the DO droplet named de-qaqc costs $16 per month and the specs are:

• 2 GB Memory
• 1 Intel vCPU
• 70 GB Disk
• Ubuntu 22.04 (LTS) x64

[sf-dcp]

Update:

Manually deployed the app from DO console and generate an app-spec.yml file that can be used for future app creation/deployment. The following roadblocks were identified:

• can't use DO firewall with app platform
• app env variables such as AWS creds to s3 should go into app-spec.yml. Need to figure out how to pass these variables to the yaml file without explicitly including variable values.
• need to figure out how to implement our security software in the app.

If variables & security software can be figured out, we can transition to Platform App and add an authentication page to streamlit.