Containerise the backend

[eternaltyro]

Containerise the backend infrastructure

Currently, the backend is run on AWS EC2 VMs. The aim of this epic is to completely containerise the backend so that it can be run on AWS ECS or equivalent service.

Aspirational Service Quality Objectives

• Open Standard / vendor neutral

  • Building containers to OCI specs - using buildah / podman
  • Standard libraries and packages

• End-to-end environment security

  • Container security scanning using Quay / Synk
  • Sane practices for injecting secrets
  • Secure container build steps
  • SBOM and license compliance
  • Trufflehog scans to check for leaked credentials

• End-to-end observability

  • Service availability monitoring
  • APM
  • Stack traces

• Seemless Scalability

  • Scale to meet demand. fast.

• Easily maintained

  • Updates, deployments and rollbacks should be easy and fast.

Tasks involved

• [ ] https://github.com/hotosm/tasking-manager/issues/5313
• [x] https://github.com/hotosm/tasking-manager/issues/5008
• [ ] https://github.com/hotosm/tasking-manager/issues/4918
• [ ] https://github.com/hotosm/tasking-manager/issues/5706
• [x] https://github.com/hotosm/tasking-manager/issues/5483
• [x] https://github.com/hotosm/tasking-manager/issues/4906
• [x] https://github.com/hotosm/tasking-manager/issues/4223
• [x] https://github.com/hotosm/tasking-manager/issues/3647
• [ ] https://github.com/hotosm/tasking-manager/issues/3636
• [ ] https://github.com/hotosm/tasking-manager/issues/3633

Pull Requests to review

• [ ] Incorporate changes from - https://github.com/hotosm/tasking-manager/pull/5642
• [ ] Incorporate changes from - https://github.com/hotosm/tasking-manager/pull/5463
• [ ] https://github.com/hotosm/tasking-manager/pull/5087
• [ ] obsolete https://github.com/hotosm/tasking-manager/pull/4857

[spwoodcock]

I would be happy to contribute a Helm chart if a Kubernetes based deployment is desirable.

But it may be best to do one thing at a time. VM --> Containers --> Orchestration System.

[eternaltyro]

Thank you, @spwoodcock for the kind offer. I agree that we should move one step at a time. Currently, I am focusing on Containers. If you think there is any other quality objective that I missed above, please feel free to add them here and we can discuss where in the roadmap it should go.