kubernetes-sigs / aws-load-balancer-controller

A Kubernetes controller for Elastic Load Balancers
https://kubernetes-sigs.github.io/aws-load-balancer-controller/
Apache License 2.0
3.9k stars 1.45k forks source link

backend-security-group not used #3669

Closed pauldtill closed 4 months ago

pauldtill commented 5 months ago

Describe the bug When we pass the --backend-security-group argument to the controller, the provided SG is used as expected on the resulting ALB, along with a dynamically created group for the frontend.

However, if we also provide this annotation, passing an existing pre-created security group ID - alb.ingress.kubernetes.io/security-groups: sg-xxxxx

The backend security group is no longer used - it only uses the group provided in this annotation.

To me at least, this contradicts the documentation, which implies that the backend security group is always used, and the annotation simply controls the frontend group.

Steps to reproduce

Expected outcome Both backend and annotation (frontend) security groups are used by the ALB.

Environment

Additional Context: Reference - https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.5/deploy/security_groups/

I can obviously workaround this, by passing both my frontend and backend SG ID's to the annotation, but this seems to defeat the object of the --backend-security-group argument, at least how I interpret it.

shraddhabang commented 5 months ago

Hey @pauldtill , Thank you for reaching out to us.

There are few things you need to consider when you are using the frontend security group using annotation alb.ingress.kubernetes.io/security-groups. As mentioned in here, If the frontend security groups are manually specified, the LBC will not by default add any rules to the backend security group. To enable managing backend security group rules automatically when using custom frontend SG, you need to apply an additional annotation to Ingress and Service resources alb.ingress.kubernetes.io/manage-backend-security-group-rules: 'true' for ingress or service.beta.kubernetes.io/aws-load-balancer-manage-backend-security-group-rules:'true' for services. However if management of backend security group rules is enabled with an annotation on a Service or Ingress, then --enable-backend-security-group must be set to true. Can you please try this out and see if this works for you?

We have explained this configuration here as well. https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.7/deploy/security_groups/#enable-autogeneration-of-backend-security-group-rules

pauldtill commented 5 months ago

hi @shraddhabang - thanks for the response.

Just to be clear on what we are trying to achieve here. We don't want the LBC to manage anything for us on the security groups, including rules.

We want to pre-create the security groups, with only the ingress/egress rules we need, per our internal security requirements, and then every LB that the LBC creates should use these.

From reading the docs, I thought the way to achieve this would be -

From what you are saying, if we want full control over the SG's used by LBC, we should simply create one SG, and pass this using the alb.ingress.kubernetes.io/security-groups annotation ? Is this correct?

Thanks again, Paul

oliviassss commented 4 months ago

From what you are saying, if we want full control over the SG's used by LBC, we should simply create one SG, and pass this using the alb.ingress.kubernetes.io/security-groups annotation ? Is this correct?

Yes, your understanding if correct, if you use self-managed SG and pass it to the ingress via alb.ingress.kubernetes.io/security-groups, the controller assumes user to manage all the SG and rules by default.