search

kubernetes-sigs / aws-load-balancer-controller

A Kubernetes controller for Elastic Load Balancers
https://kubernetes-sigs.github.io/aws-load-balancer-controller/
Apache License 2.0
3.82k stars 1.41k forks source link

Not able to modify schema from internet-facing to internal if ALB was created with internet-facing at the first place #3729

Closed bryanfang closed 2 weeks ago

bryanfang commented 1 month ago

Describe the bug I deployed our stack into EKS and created a internet-facing ALB through alb annotation via terraform code, the version of controller is v2.7 alb.ingress.kubernetes.io/scheme: "internet-facing" now I want to change the schema from internet-facing to internal, but controller will show reconcile error "conflicting scheme: map[internal:{} internet-facing:{}]" this issue won't happen if ALB was created directly via terraform code without ALB-Controller and Annotation

Steps to reproduce

  1. deploy applications and ingress via terraform code. special annotation note: alb.ingress.kubernetes.io/scheme: "internet-facing"
  2. change schema to internal and redeploy the terraform code
  3. check schema type of ALB, it's still internet-facing
  4. check logs of alb-controller, it will show above error message

Expected outcome Schema type of ALB should be consistent with code

Environment

Additional Context: NA

wweiwei-li commented 3 weeks ago

Can you confirm if you are setting the scheme at a ingressClassParam level, and fall into this scenario: https://github.com/kubernetes-sigs/aws-load-balancer-controller/blob/e5d625f96415fd44e6399e9c75e2bd985f5a2288/pkg/ingress/model_build_load_balancer.go#L144C29-L144C48

wweiwei-li commented 3 weeks ago

More reference : https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.8/guide/ingress/ingress_class/#specscheme:~:text=spec.-,scheme,-%C2%B6

wweiwei-li commented 3 weeks ago

More reference : https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.6/guide/ingress/annotations/#scheme:~:text=Exclusive%3A%20such%20annotation%20should%20only%20be%20specified%20on%20a%20single%20Ingress%20within%20IngressGroup%20or%20specified%20with%20same%20value%20across%20all%20Ingresses%20within%20IngressGroup.

bryanfang commented 3 weeks ago

ingress: enabled: true annotations: kubernetes.io/ingress.class: "alb" alb.ingress.kubernetes.io/group.name: ${stage}-monitoring-ingress alb.ingress.kubernetes.io/load-balancer-name: ${stage}-monitoring-elb-${random_lb_suffix} alb.ingress.kubernetes.io/security-groups: ${sg_monitoring_id} alb.ingress.kubernetes.io/manage-backend-security-group-rules: "true" alb.ingress.kubernetes.io/ssl-redirect: '443' alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}' alb.ingress.kubernetes.io/ip-address-type: ipv4 alb.ingress.kubernetes.io/scheme: "internal" alb.ingress.kubernetes.io/target-type: "ip" alb.ingress.kubernetes.io/healthcheck-path: /alertmanager/-/healthy alb.ingress.kubernetes.io/listen-ports: ${alb_listener_ports} alb.ingress.kubernetes.io/ssl-policy: "ELBSecurityPolicy-TLS13-1-2-2021-06" alb.ingress.kubernetes.io/tags: Environment=${stage},Team=SRE,Purpose=Monitoring`

this is my configuration

wweiwei-li commented 3 weeks ago

If group.name specified, all Ingresses belong to the same IngressGroup specified will result in a single ALB. Please make sure there are no scheme conflicts within all ingresses in the same group.

bryanfang commented 2 weeks ago

@wweiwei-li thanks for pointing me to the right direction, and I found I have to manually remove all original ingress and rename the new ingress https://github.com/kubernetes/kubernetes/issues/95983, then it will work