IAC: dry k8s service templates

ranchodeluxe commented 1 year ago

What I am changing

DRY up service templates and values.yaml where necessary: https://github.com/developmentseed/eoAPI/issues/80 (NOTE: my gut is telling me this isn't too ugly but wondering what other folks think 😄 )
add db.settings.resources.requests.storage as a value
remove any gunicorn-specific os environ variables
move to using envFrom
clean up validation even though we eventually will move to validation in JSON Schema via https://github.com/developmentseed/eoAPI/issues/88

How I did it

Being a yaml programmer is a thankless and boring job 😉 🍷

How you can test it

Good point, we'll need tests at some point too. If you have access to a cluster run:

  $ export GITSHA=$(git rev-parse HEAD | cut -c1-10)
  $ export PGUSER_VAR=s00pers3cr3t
  $ export POSTGRES_USER_VAR=s00pers3cr3t
  $ export POSTGRES_PASSWORD_VAR=superuserfoobar
  $ export PGPASSWORD_VAR=foobar

  $ helm install \
      --namespace eoapi \
      --create-namespace \
      --set gitSha=$GITSHA \
      --set db.settings.secrets.PGUSER=$PGUSER \
      --set db.settings.secrets.POSTGRES_USER=$POSTGRES_USER \
      --set db.settings.secrets.PGPASSWORD=$PGPASSWORD \
      --set db.settings.secrets.POSTGRES_PASSWORD=$POSTGRES_PASSWORD \
      eoapi \
      ./eoapi

batpad commented 1 year ago

@ranchodeluxe this is looking really great to me.

I like the idea of "DRY"ing things up here and looping through the different services.

The single manifest file though does seem like it gets a bit big and slightly hard to follow. What do you think about splitting things up into like deployments.yaml, services.yaml, configmaps.yaml, etc. to keep each type of resource in its own file and loop?

Not a strong opinion though - if this seems nicer for you, works for me. Left a comment on repeating the PGPASSWORD multiple times in values. That would be really nice to fix if possible? Possibly by mapping the same value to multiple items in the ConfigMap or so?

batpad commented 1 year ago

(NOTE: my gut is telling me this isn't too ugly but wondering what other folks think 😄 )

Haa - it may also be a terrible idea :D - recently did something roughly similar here: https://github.com/IFRCGo/go-api/blob/terraform-prod/deploy/helm/ifrcgo-helm/templates/cronjobs/jobs.yaml - defining multiple cronjobs in values and then looping through them. Figuring out this range stuff and looping in the YAML felt a bit awful, but I think it's nice, as long as you're sure the different services are similar enough and won't require special casing within the loop?

batpad commented 1 year ago

Good point, we'll need tests at some point too.

@ranchodeluxe would love to see what our strategies around testing helm charts can be - we have a few tickets to figure this out and I had spent some time a while ago trying to figure running k3s in CI and installing the helm chart and running tests, but there was just too much pain and we didn't have enough time. Would b super great to see a good pattern to do this and it would be useful for us across a few projects where we produce helm charts.

ranchodeluxe commented 1 year ago

(NOTE: my gut is telling me this isn't too ugly but wondering what other folks think 😄 )

Haa - it may also be a terrible idea :D - recently did something roughly similar here: https://github.com/IFRCGo/go-api/blob/terraform-prod/deploy/helm/ifrcgo-helm/templates/cronjobs/jobs.yaml - defining multiple cronjobs in values and then looping through them. Figuring out this range stuff and looping in the YAML felt a bit awful, but I think it's nice, as long as you're sure the different services are similar enough and won't require special casing within the loop?

Yeah, it is awful to figure out but now we're helm magicians and somewhere on our tombstones it will say "advanced yaml programmer" 😞

ranchodeluxe commented 1 year ago

@ranchodeluxe this is looking really great to me.

I like the idea of "DRY"ing things up here and looping through the different services.

The single manifest file though does seem like it gets a bit big and slightly hard to follow. What do you think about splitting things up into like deployments.yaml, services.yaml, configmaps.yaml, etc. to keep each type of resource in its own file and loop?

Not a strong opinion though - if this seems nicer for you, works for me. Left a comment on repeating the PGPASSWORD multiple times in values. That would be really nice to fix if possible? Possibly by mapping the same value to multiple items in the ConfigMap or so?

Yeah, @ividito and I were talking about the same things. Four resources is small enough I think to keep as is in a single template. Creating that loop in multiple files makes me want to die a little

ranchodeluxe commented 1 year ago

Good point, we'll need tests at some point too.

@ranchodeluxe would love to see what our strategies around testing helm charts can be - we have a few tickets to figure this out and I had spent some time a while ago trying to figure running k3s in CI and installing the helm chart and running tests, but there was just too much pain and we didn't have enough time. Would b super great to see a good pattern to do this and it would be useful for us across a few projects where we produce helm charts.

The basic idea is that we just need to template out a few things and then use yq and bash assertions. But I'm shopping for GH actions that might help. They all seem pretty weak sauce and don't test outputs which IMHO is the most important thing

developmentseed / eoAPI