search

kubernetes-sigs / aws-load-balancer-controller

A Kubernetes controller for Elastic Load Balancers
https://kubernetes-sigs.github.io/aws-load-balancer-controller/
Apache License 2.0
3.96k stars 1.47k forks source link

External load balancer via targetGroupARN does not modify the security group to allow traffic from load balancer #3703

Closed prakashbalaji closed 1 month ago

prakashbalaji commented 6 months ago

Describe the bug The pods are getting registered to the targetGroupARN specified correctly but the. security groups in the nodes are not adjusted to allow the traffic from the load balancer. Even specifically passing security group with ingress annotation is not helping.

Is this an bug here? if this is not a bug what is the correct way to adjust the node security group to allow traffic from load balancer for external load balancer.

Steps to reproduce

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-app
  labels:
    app: nginx-app
spec:
  selector:
    matchLabels:
      app: nginx-app
  replicas: 2
  template:
    metadata:
      labels:
        app: nginx-app
    spec:
      containers:
      - name: nginx-container
        image: nginx
        ports:
            - containerPort: 80        
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
  labels:
    app: nginx-app
  annotations:
    alb.ingress.kubernetes.io/healthcheck-path: /index.html    
spec:
  type: ClusterIP
  selector:
    app: nginx-app
  ports:
    - port: 80
      targetPort: 80
---

apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata:
  name: nginx-manual-tg
spec:
  serviceRef:
    name: nginx-service
    port: 80
  targetGroupARN: arn:aws:elasticloadbalancing:us-west-2:<AWS_ACCOUNT_ID>:targetgroup/nginx-manual-tg/<HASH>

Expected outcome Should security group of node be adjusted to allow traffic from load balancer for external load balancer?

Environment

AWS Load Balancer controller version : v2.7.2 Kubernetes version v1.29.3-eks-adc7111 Using EKS (yes/no), if so version? yes - v1.29.3-eks-adc7111

Additional Context:

For cases when the load balancer is created by the controller, the node security groups are adjusted correctly to allow traffic from load balancer.

oliviassss commented 6 months ago

@prakashbalaji hey, was the load balancer created successfully? were you trying to create a NLB or ALB?

prakashbalaji commented 6 months ago

@oliviassss - This is the case of we creating load balancer externally outside of the controller. We used terraform to provision the load balancer and we are using ALB here. The issue is that the pods are getting registered as targets to the load balancer but the EKS node security group rules are not amended to allow traffic from the load balancer.

When the controller creates the load balancer it augments the security group rules.

oliviassss commented 6 months ago

@prakashbalaji, sorry I missed that. I doubt if it's an external LB, the controller will manage the AWS resources for it (since we rely on users to manage TG, listener, LB for external LB). Personally I haven't tested this before, I can try to repro, but check out the live doc here: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.8/guide/use_cases/self_managed_lb/

prakashbalaji commented 6 months ago

Thanks @oliviassss for the response, I can for sure tell you that the security group rules are not amended for the self managed lb. As I think more, it seems right for the controller not to manage the security group as we create the load balancer and security group outside and we are supposed to allow traffic from load balancer to EKS node, I think this issue can be closed as we are taking that approach and also it would be good to document it in the notes about security groups. The linked article does not talk about security groups and hence the confusion. Thanks for your inputs btw.

oliviassss commented 6 months ago

Thanks for the confirmation. /kind documentation

M00nF1sh commented 6 months ago

@prakashbalaji It's actually possible for a TargetGroupBinding to modify the worker node security groups. check out the spec.networking on TargetGroupBinding. It supports to allow reach target on specific ports from specified cidr or security group(preferred to use the LB's security Group)

k8s-triage-robot commented 3 months ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot commented 2 months ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot commented 1 month ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

k8s-ci-robot commented 1 month ago

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to [this](https://github.com/kubernetes-sigs/aws-load-balancer-controller/issues/3703#issuecomment-2424328665): >The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. > >This bot triages issues according to the following rules: >- After 90d of inactivity, `lifecycle/stale` is applied >- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied >- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed > >You can: >- Reopen this issue with `/reopen` >- Mark this issue as fresh with `/remove-lifecycle rotten` >- Offer to help out with [Issue Triage][1] > >Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). > >/close not-planned > >[1]: https://www.kubernetes.dev/docs/guide/issue-triage/ Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.