P0NDER0SA commented 3 days ago

Description

As a developer of notify, I would like all of our kubernetes deployments to be managed by Helmfile so that all of our kubernetes code is in one location. I would also like to have all secrets automated from a single source of truth (1pass -> AWS Secrets) and remove our dependency on encrypted .env files

WHY are we building?

Provides the ability to deploy dynamic environments
Provides the ability to run "diff" commands against environments
Reduces the amount of code that Notify has to manage
Keeps code DRY

WHAT are we building?

[ ] Add Document Download entry to the helmfile
[ ] source secrets for helmfile from AWS Secrets Manager
[ ] source configuration values from helmfile overrides where necessary
[ ] adjust GitHub actions to work with these changes
[ ] This work is scoped to the DEV environment.

VALUE created by our solution

Increased reliability via reduced chance of accidental changes during releases
Increased velocity due to reduction in maintenance
better secrets management that falls in line with our OKRs

Acceptance Criteria

[ ] Celery Main are deployed and managed with helmfile
[ ] All secrets required by Document Download are sourced from 1pass -> terraform -> aws secrets manager and accessed using a kubernetes secret
[ ] Celery Main still works

Not in scope

[ ] Deployment in staging and production: we should only deploy to dev until we figure out the deployment steps.

QA Steps

[ ] verify that the Celery Pods are functional and running properly (might need some guidance on the best way to QA this!)

P0NDER0SA commented 3 days ago

Working on the migration for this one, which includes 3 deployments. it's been deployed -- but isn't working yet. Need to migrate the rest of the Env vars and secrets, and then debug the deployment.

https://github.com/cds-snc/notification-manifests/pull/3103

P0NDER0SA commented 2 days ago

adding IAM https://github.com/cds-snc/notification-terraform/pull/1661/files

sastels commented 2 days ago

Work progressing on the main part of celery. Will get together with Ben re some details, in particular reading secrets.

sastels commented 1 day ago

Chatted with Ben on streamlining reading secrets.

P0NDER0SA commented 20 hours ago

PR to tighten up the other components using the same approach before moving on to the last couple of kustomize portions that we need to migrate to helm

https://github.com/cds-snc/notification-manifests/pull/3126/files

cds-snc / notification-planning-core

Move Celery Main to Helmfile/AWS Secrets Manager #457