MEDIUM: Single ALB with Path Rewrites via Ingress Nginx

[ranchodeluxe]

Background:

What we know:

• We can group kind: Ingress for each kind: Service onto a shared ALB to save monies

• We can then use path-based matching to forward traffic to the correct service

• ALB(s) still (b/c the last time i ran into this was 2020) don't have the ability to forward traffic AND also do path rewriting like a proxy would. Huge bummer IMHO

• While there are workarounds via Lambda we don't want to have to think about those limitations too: cold starts, invocation limits, latency etc

What the above means in practice is that a request that comes in on the main ALB /vector cannot have the prefix stripped and will actually get sent to the tipg service as /vector and then 404

What we want to try:

• Instead of using the AWS Load Balancer Controller lets try using Ingress Nginx Controller and get the path rewriting we want

AC:

• [ ] review Ingress Nginx and how to get set up (pay close attention to setting up the IAM <> k8s ServiceAccount bridge via OIDC since that's how anything in k8s tells AWS what to do dynamically for EBS volume mounts, ALB/NLB(s), ENI(s))
• [ ] document steps for IAC Terraform or eksctl later
• [ ] figure out how our path rewriting would work in this setting
• [ ] keep the existing options in the templates and values.yaml (enable_shared_ingress) so folks can choose whether they want path-based routing and single shared ALB or a single ALB per service

[ranchodeluxe]

What the above means in practice is that a request that comes in on the main ALB /vector cannot have the prefix stripped and will actually get sent to the tipg service as /vector and then 404

based on this sentence above I think using --root-path override on each service would also work with the ALB approach