Which annotations are needed to use multiple ingresses in a group?

[johnjeffers]

Apologies for asking a question instead of reporting an issue, but this isn't answered in the docs anywhere that I can find.

When you are creating a group of ingresses with alb.ingress.kubernetes.io/group.name what annotations are required to be present on all of the ingresses in the group?

From my testing, it looks like you need:

    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/group.name: my-group
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP":80},{"HTTPS":443}]'

If I'm missing one of those annotations, the target group rules don't get created correctly.

I would appreciate if someone could confirm that.

[johnjeffers]

I should probably ask this question more clearly.

If I have multiple ingresses in a group, do I have to set every single annotation on every single ingress? Or can I have a "default" ingress that has the majority of the annotations I need, while the other ingresses simply inherit from that one?

Let's say I have a "default" ingress that looks like this:

    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/group.name: my-group
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP":80},{"HTTPS":443}]'
    alb.ingress.kubernetes.io/tags: environment=dev
    alb.ingress.kubernetes.io/load-balancer-attributes: deletion_protection.enabled=true
    alb.ingress.kubernetes.io/certificate-arns: my-cert-arns
    alb.ingress.kubernetes.io/security-groups: my-sg
    alb.ingress.kubernetes.io/wafv2-acl-arn: my-waf-arn

Which of those annotations need to be present on other ingresses in the same group?

As I mentioned above, it looks like all I need is:

    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/group.name: my-group
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP":80},{"HTTPS":443}]'

and the other annotations are "inherited" from what I'm calling the default ingress.

[M00nF1sh]

@johnjeffers thanks for creating this issue(and feel free to do it as it means our docs isn't good) 😄

It's currently documented as "MergeBehavior" column in the annotation table. In general,

• annotation based settings specific to LoadBalancer are MergeBehavior "Exclusive", which means you should only apply on a single Ingress within group. (we also accept if you add the same annotation with same value on every Ingress). e.g. alb.ingress.kubernetes.io/scheme, alb.ingress.kubernetes.io/load-balancer-attributes, alb.ingress.kubernetes.io/security-groups and alb.ingress.kubernetes.io/wafv2-acl-arn.

other annotations are MergeBehavior "Merge". which means they will impact each Ingress separately, and the semantic slightly differs per annotation.

• alb.ingress.kubernetes.io/listen-ports controls which port will be ingress rule in a Ingress be assigned to the LoadBalancer, so it needs to be on each Ingress. (so that you have a Ingress contains rules only for port 80 and another Ingress contains rules only for port 443 on a single LoadBalancer).
• alb.ingress.kubernetes.io/certificate-arns controls the certificate for your Ingress. each Ingress must define their own certificate even it's within a group. Otherwise, the controller will try to auto-discover a certificate for the hostname your Ingress uses(no matter whether other Ingress within IngressGroup defined certificate or not). we'll improve this behavior soon: https://github.com/kubernetes-sigs/aws-load-balancer-controller/issues/1776
• alb.ingress.kubernetes.io/tags is merged together.(semantic differs per AWS resource we created): tags on Loadbalancer/SecurityGroup are using the merged tags on all Ingresses, while tags on TargetGroup only uses tags defined on individual Ingress/Service)

[johnjeffers]

Thank you for the explanation, super helpful. I'm embarrassed that I missed the stuff on merge behavior in the docs 😳