This is as embarrassingly huge PR. Sorry for that.
Contained within are the additions and changes I needed to make in order to get Access the Data ready to deploy. Note well! AtD isn't successfully deployed yet, but that has to do with issues with CKAN, not the incubator platform.
Introduction
(yeah, it's that kind of PR)
As a rough guide to what's changed:
The headline is the multi-container-service module. It's designed with AtD in mind, but in general I suspect that being able to deploy e.g. a web backend and Redis (say) will be really valuable to Hack for LA. MCS also covers details like setting up public and private DNS. MCS covers a couple other outstanding Incubator issues, c.f. #18
MCS also drives an ecr module, to create container registries that developers can push to to update their services.
Other new modules include database which uses the cyrilgdn/postgresql provider to provision postgres databases and roles within the shared RDS instance. All credentials are stored in SSM parameters as SecureStrings, so there's no toxic secrets to manage in IaC. (c.f. #1)
Because that's such a useful pattern, there's also a cheap-secrets module, which allows for purpose-agnostic secrets to be securely generated and stashed in SSM. This isn't 100% ideal, but the benefit is that managing secrets this way is free, as compared to using AWS SecretStore. The downside is that the secret material might be stored in terraform state files and would theoretically be extractable. The same personnel who could do that, though, could simply query SSM, or deploy a task that would forward secrets to them.
The Terraform shared_resources modules have been fleshed out significantly, such that we can get data from all of them.
Overall, I feel very good about these changes, and would recommend that future Incubator projects be built using multi-container-service on AtD's pattern even if they only deploy a single container.
This is as embarrassingly huge PR. Sorry for that.
Contained within are the additions and changes I needed to make in order to get Access the Data ready to deploy. Note well! AtD isn't successfully deployed yet, but that has to do with issues with CKAN, not the incubator platform.
Introduction
(yeah, it's that kind of PR)
As a rough guide to what's changed: The headline is the
multi-container-servicemodule. It's designed with AtD in mind, but in general I suspect that being able to deploy e.g. a web backend and Redis (say) will be really valuable to Hack for LA. MCS also covers details like setting up public and private DNS. MCS covers a couple other outstanding Incubator issues, c.f. #18MCS also drives an
ecrmodule, to create container registries that developers can push to to update their services.Other new modules include
databasewhich uses thecyrilgdn/postgresqlprovider to provision postgres databases and roles within the shared RDS instance. All credentials are stored in SSM parameters asSecureStrings, so there's no toxic secrets to manage in IaC. (c.f. #1)Because that's such a useful pattern, there's also a
cheap-secretsmodule, which allows for purpose-agnostic secrets to be securely generated and stashed in SSM. This isn't 100% ideal, but the benefit is that managing secrets this way is free, as compared to using AWS SecretStore. The downside is that the secret material might be stored in terraform state files and would theoretically be extractable. The same personnel who could do that, though, could simply query SSM, or deploy a task that would forward secrets to them.The Terraform
shared_resourcesmodules have been fleshed out significantly, such that we can get data from all of them.Overall, I feel very good about these changes, and would recommend that future Incubator projects be built using
multi-container-serviceon AtD's pattern even if they only deploy a single container.