In order to reduce the cost of operating EKS cluster instances, and reduce dependence on a single k8s provider, the team would like to provision just a single ALB per AWS EKS cluster, rather than one per individual ingress.
Acceptance Criteria
[ACs should be clearly demoable/verifiable whenever possible. Try specifying them using BDD.]
[x] GIVEN I have provisioned an EKS instance \
AND I have deployed two ingresses \
AND I am authenticated with the AWS console \
WHEN I look at the AWS EC2 "Load Balancer" list \
THEN I see just one LB associated with the EKS cluster
Background
People using cloud.gov to provision and bind k8s service should, as much as possible, not care about and not refer to the underlying implementation when they use the service. This leaves the provider of the k8s service flexibility to use a different implementation (eg GCP or Azure instead of AWS) without customers of the service preventing migration.
Normally, using AWS EKS requires customers to know about and use the AWS-specific ingress annotations in order to make their deployments accessible to the outside world. By using a second level ingress controller based on the widely used and documented nginx-ingress, we can enable cross-provider specifications on ingress using labels and tags that will not need to be changed.
Using a secondary controller has the added benefit of requiring just a single AWS Load Balancer instance for all the workloads in the cluster, no matter how many. This in turn cuts down on the government's cost to run the service.
The secondary nginx-ingress controller is only accessible from within Fargate, and for traffic to reach that service, it must traverse the AWS ALB first. So no additional network exposure is implied here.
Because customers of the service cannot specify tags or labels that the ALB controller will act on, there is no way for customers to introduce a separate ingress to the cluster.
User Story
In order to reduce the cost of operating EKS cluster instances, and reduce dependence on a single k8s provider, the team would like to provision just a single ALB per AWS EKS cluster, rather than one per individual ingress.
Acceptance Criteria
[ACs should be clearly demoable/verifiable whenever possible. Try specifying them using BDD.]
Background
People using cloud.gov to provision and bind k8s service should, as much as possible, not care about and not refer to the underlying implementation when they use the service. This leaves the provider of the k8s service flexibility to use a different implementation (eg GCP or Azure instead of AWS) without customers of the service preventing migration.
Normally, using AWS EKS requires customers to know about and use the AWS-specific ingress annotations in order to make their deployments accessible to the outside world. By using a second level ingress controller based on the widely used and documented nginx-ingress, we can enable cross-provider specifications on ingress using labels and tags that will not need to be changed.
Using a secondary controller has the added benefit of requiring just a single AWS Load Balancer instance for all the workloads in the cluster, no matter how many. This in turn cuts down on the government's cost to run the service.
Security Considerations (required)
The secondary nginx-ingress controller is only accessible from within Fargate, and for traffic to reach that service, it must traverse the AWS ALB first. So no additional network exposure is implied here.
Because customers of the service cannot specify tags or labels that the ALB controller will act on, there is no way for customers to introduce a separate ingress to the cluster.
Sketch
There is precedent for setting up this architecture with AWS EKS and Fargate.