Epic: Scooze as a Service (Saas)

[cmentzer]

Providing Scooze as a publicly / semi-publicly available service

This ticket is just a starting point to enumerate at a high level what needs to be done to support running scooze as a hosted service (probably on AWS).

We should use IaC. SBWH (strong belief weakly held) we should use either Terraform or, if we want to be cool, Pulumi

• Either way, we should decide if that code lives in this repo, or a separate repo. I'd advocate for a separate repo, personally.
• Pulumi is pretty neat. I'd love an excuse to mess around with it, and it might be easier to pick up for the non-infra folks on the project.

Rough list of infrastructure requirements

• Network components:
  • vpc / vnet & subnets, security group rules
  • Load balancer(s)
  • DNS
    • We should probably buy or use one of our existing domains, also
• Compute components:
  • Personally I'd suggest ECS, as it provides a reasonable middle ground between EC2 and EKS. If we want to start on just EC2 that makes plenty of sense, and if we want to be provide agnostic then we could consider EKS, but it's a PITA to get started on
• Data layer
  • Mongo or SQL, either way AWS has a hosted option. Aurora is quite good, or Atlas for Mongo.
• Connective tissue:
  • regardless of platform, there's a bunch of random stuff in IAM roles and policies, security groups on subnets, log forwarding, etc.; that needs to be setup before we can really "go live"

CI/CD

• We should build and push container images on any merge to our "main" branch, either to ECR or docker hub directly
• We should be able to integrate GHA into AWS and have GHA handle updating our compute resources with new container images automatically

[iambroadband]

I think this is a good conversation to start because the best improvements we've had to our existing code have come from outside influence. When we use our own tools to build other applications, it's helped us recognize where the value truly lies and what features need to be changed or added. I'll try to address everything in the OP point by point.

IaC

I think using Pelumi sounds cool since we are starting fresh and it sounds like you want to get your hands on it. If you think it's a reasonably good fit for the job, I see no problem going ahead with something that isn't Terraform.

Network Components

Agree on all points. re: domain, I may have already purchased one some time ago, but I forget now. If not, we can buy one for this purpose, seems worth it to me.

Compute Components

I'd like to either use ECS or EC2 here. I think EKS is overkill. Also, is there a reason we couldn't do most of our compute with API Gateway and Lambda? Do we need persistent compute?

Data Layer

I think we've agreed to stick with Mongo for now to avoid complicated backend changes. We don't expect the performance to be an issue at our scale, but if things change, of course we can revisit. Atlas it is.

Connective Tissue

This is the bit that I'm least familiar with, so I will defer to you.

CI/CD

Is there a way to publish multiple versions of a container? I don't want to publish dev as latest and we don't use a main branch that tracks only releases. Ideally we would push a docker container to latest whenever we pushed tags to our repo (cut a release in git). For situations where we merge to dev and we want a dev container, can we have a separate version for that?