kubernetes-sigs / aws-load-balancer-controller

A Kubernetes controller for Elastic Load Balancers
https://kubernetes-sigs.github.io/aws-load-balancer-controller/
Apache License 2.0
3.93k stars 1.46k forks source link

[Question] Manually set a rule in the ALB #3782

Closed RobinFrcd closed 2 months ago

RobinFrcd commented 3 months ago

Hi, I'd like to manually add a rule to an ALB managed by the controller. I've seen related issues like https://github.com/kubernetes-sigs/aws-load-balancer-controller/issues/3763, but not sure about the conclusion. Is there a way, with tags or something else, to ask the controller not to delete a rule I've created manually. I have a service running outside EKS, on an EC2 instance, that I want to expose through the ALB.

Thank you very much !

wweiwei-li commented 2 months ago

Our controller by design will delete resources you created manually. We will suggest not doing manual modification on it.

wweiwei-li commented 2 months ago

Same issue ad : https://github.com/kubernetes-sigs/aws-load-balancer-controller/issues/3763

MartinEmrich commented 2 months ago

@RobinFrcd you can have aws-load-balancer-controller create the rule for you, like this:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-ec2-service
  annotations:
    alb.ingress.kubernetes.io/group.name: whatever
    alb.ingress.kubernetes.io/group.order: "123"
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
    alb.ingress.kubernetes.io/actions.forward-my-ec2: >
      {"type":"forward","targetGroupARN": "<arn-of-your-targetgroup>"}
    alb.ingress.kubernetes.io/healthcheck-protocol: HTTP
    alb.ingress.kubernetes.io/healthcheck-port: "8000" # whatever
    alb.ingress.kubernetes.io/healthcheck-path: "/" # whatever
    alb.ingress.kubernetes.io/success-codes: "200"
spec:
  ingressClassName: alb
  rules:
    http:
      paths:
      - path: /my-service-on-ec2-whatever
        pathType: Prefix
        backend:
          service:
            name: forward-my-ec2
            port:
              name: use-annotation
modevops commented 3 weeks ago

This a really bad answer for me. I am trying to do path based routing between eks and and ec2 instance by deleting manually added resources it defeats the purpose of having a customer mananged loadbalancer. Alb ingress should not delete manually added resources on a customer managed loadbalancer. It should not delete the listener. Right now it impossible to share an alb between eks and ec2 instance. To me it is very poor design. Why will not fix this issue.

MartinEmrich commented 3 weeks ago

@modevops A counterargument: I actually enjoy and expect the declarative and deterministic nature of the AWS load balancer controller: Whad I write in Ingress resources, is exactly what will happen. If you let the aws-load-balancer-controller create an ALB, it is under it's control, and not yours. So if you meddle with it, it of course will clean up. (And to be sure, there are no "non-customer-managed" ALBs, but that's not the point. you as the customer make the decision to use the AWS load balancer controller and have it create and manage ALBs on your behalf).

As ALB rules are ordered, there would be no way for the ALB controller to know and order the rules you made directly vs. the rules it generated (unless a complicated set of tags and rules is introduced).

But you still can have rules and listeners pointing to an external target group, see my example above. Via annotations, aws-load-balancer-controller more or less exposes any feature of ALB (we even use the arguably limited and finnicky OIDC authentication feature in one project successfully).