kubernetes-sigs / aws-load-balancer-controller

A Kubernetes controller for Elastic Load Balancers
https://kubernetes-sigs.github.io/aws-load-balancer-controller/
Apache License 2.0
3.93k stars 1.46k forks source link

Unable to configure group.name correctly #3098

Closed esteban1983cl closed 1 year ago

esteban1983cl commented 1 year ago

Describe the bug I want configure multiple ingress using one ALB using group.name and group.order annotations, the first ingress works perfectly but when I want add other ingress I get 404 response.

Steps to reproduce 1.- Configure IAM Role with service account in kubernetes with proper permissions. 2.- Install application load balancer controller using helm (Configured IngressClass and ingressClassParams)

apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  labels:
    app.kubernetes.io/instance: ops-awslbc-private
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: aws-load-balancer-controller-private
    app.kubernetes.io/version: v2.4.7    
    helm.sh/chart: aws-load-balancer-controller-1.4.8
  name: awslbc-private
spec:
  controller: ingress.k8s.aws/alb
  parameters:
    apiGroup: elbv2.k8s.aws
    kind: IngressClassParams
    name: awslbc-private
---
apiVersion: elbv2.k8s.aws/v1beta1
kind: IngressClassParams
metadata:
  labels:
    app.kubernetes.io/instance: ops-awslbc-private
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: aws-load-balancer-controller-private
    app.kubernetes.io/version: v2.4.7
    helm.sh/chart: aws-load-balancer-controller-1.4.8
  name: awslbc-private
spec:
  group:
    name: awslbc-private
  ipAddressType: ipv4
  scheme: internal

3.- Deploy ingresses and services with NodePort configuration.

---
# First one, works very well
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    alb.ingress.kubernetes.io/actions.ssl-redirect: >-
      {"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port":
      "443", "StatusCode": "HTTP_301"}}
    alb.ingress.kubernetes.io/backend-protocol: HTTP
    alb.ingress.kubernetes.io/certificate-arn: >-
      arn:aws:acm:us-east-1:999999999999:certificate/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
    alb.ingress.kubernetes.io/load-balancer-attributes: >-
      access_logs.s3.enabled=true,access_logs.s3.bucket=mylogsbucket,access_logs.s3.prefix=eks-ops-private,deletion_protection.enabled=true,idle_timeout.timeout_seconds=1800,routing.http.preserve_host_header.enabled=true
    alb.ingress.kubernetes.io/load-balancer-name: eks-ops-private
    alb.ingress.kubernetes.io/shield-advanced-protection: 'true'
    alb.ingress.kubernetes.io/ssl-policy: ELBSecurityPolicy-FS-1-2-Res-2020-10
    alb.ingress.kubernetes.io/ssl-redirect: '443'
    alb.ingress.kubernetes.io/subnets: 'subnet-1,subnet-2,subnet-3'
    alb.ingress.kubernetes.io/target-group-attributes: >-
      load_balancing.algorithm.type=least_outstanding_requests,stickiness.enabled=true,stickiness.type=app_cookie,stickiness.app_cookie.cookie_name=konga-test,stickiness.app_cookie.duration_seconds=180
    alb.ingress.kubernetes.io/target-type: instance
    external-dns.alpha.kubernetes.io/aws-weight: '1'
    external-dns.alpha.kubernetes.io/set-identifier: ops
  name: first-ingress-alb
  namespace: kong
spec:
  ingressClassName: awslbc-private
  rules:
    - host: first-one.ops.example.com
      http:
        paths:
          - backend:
              service:
                name: first-one
                port:
                  number: 1337
            path: /
            pathType: Prefix
    - host: first-one.example.com
      http:
        paths:
          - backend:
              service:
                name: konga
                port:
                  number: 1337
            path: /
            pathType: Prefix
---
# Second one doesn't work
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    alb.ingress.kubernetes.io/group.order: '10'
    external-dns.alpha.kubernetes.io/aws-weight: '1'
    external-dns.alpha.kubernetes.io/set-identifier: ops
  labels:
    app: cerebro
    argocd.argoproj.io/instance: ops-cerebro
    chart: cerebro-2.0.3
    heritage: Helm
    release: ops-cerebro
  name: ops-cerebro
  namespace: logging
spec:
  ingressClassName: awslbc-private
  rules:
    - host: cerebro.ops.example.com
      http:
        paths:
          - backend:
              service:
                name: ops-cerebro
                port:
                  name: http
            path: /
            pathType: ImplementationSpecific
    - host: cerebro.example.com
      http:
        paths:
          - backend:
              service:
                name: ops-cerebro
                port:
                  name: http
            path: /
            pathType: ImplementationSpecific
---
# Third one doesn't work
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    alb.ingress.kubernetes.io/group.order: '20'
    external-dns.alpha.kubernetes.io/aws-weight: '1'
    external-dns.alpha.kubernetes.io/set-identifier: ops
  labels:
    argocd.argoproj.io/instance: ops-bookstack
    k8s-app: bookstack
  name: bookstack-ingress-nginx
  namespace: bookstack
spec:
  ingressClassName: awslbc-private
  rules:
    - host: bookstack.ops.example.com
      http:
        paths:
          - backend:
              service:
                name: bookstack
                port:
                  number: 8001
            path: /
            pathType: ImplementationSpecific

curl -IL https://bookstack.ops.example.com returns a 404 response code the dns names are registered using external-dns addon

Expected outcome I need that ALB supports theses additionals host names. Unable to see them aws console, only I see the first one.

Environment

Additional Context: N/A

riita10069 commented 1 year ago

You have to add group.name annotation like this

    alb.ingress.kubernetes.io/group.name: 'sample'
    alb.ingress.kubernetes.io/group.order: '10'
    alb.ingress.kubernetes.io/group.name: 'sample'
    alb.ingress.kubernetes.io/group.order: '20'
    alb.ingress.kubernetes.io/group.name: 'sample'
    alb.ingress.kubernetes.io/group.order: '30'
esteban1983cl commented 1 year ago

Hi @riita10069 it doesn't work :(

kishorj commented 1 year ago

@riita10069, you could specify group name via IngressClassParams as well. @esteban1983cl, for the ingress you see 404 response, could you make sure the backend service exists in your namespace. Controller adds a 404 rule if it is unable to find the backend service - you could verify this from the controller logs.

esteban1983cl commented 1 year ago

Hi everyone, for future googlers, it's necessary add the follow annotation by ingress for handle differents ingress and one ALB

alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'

  annotations:
    external-dns.alpha.kubernetes.io/set-identifier: "ops"
    external-dns.alpha.kubernetes.io/aws-weight: "1"
    alb.ingress.kubernetes.io/group.order: '20'
    alb.ingress.kubernetes.io/group.name: 'eks-ops-private'
    alb.ingress.kubernetes.io/ssl-redirect: '443'
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
    alb.ingress.kubernetes.io/healthcheck-path: /login
jhughes-mc commented 1 year ago

Hi everyone, for future googlers, it's necessary add the follow annotation by ingress for handle differents ingress and one ALB

alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'

  annotations:
    external-dns.alpha.kubernetes.io/set-identifier: "ops"
    external-dns.alpha.kubernetes.io/aws-weight: "1"
    alb.ingress.kubernetes.io/group.order: '20'
    alb.ingress.kubernetes.io/group.name: 'eks-ops-private'
    alb.ingress.kubernetes.io/ssl-redirect: '443'
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
    alb.ingress.kubernetes.io/healthcheck-path: /login

This resolved the issue for me. I was not defining alb.ingress.kubernetes.io/listen-ports in my other ingress group members and as a result the rules were getting deleted.

oliviassss commented 1 year ago

I'm closing this issue as for now, please feel free to reach out or reopen if you have any questions. Thanks.