kubernetes / cloud-provider-aws

Cloud provider for AWS
https://cloud-provider-aws.sigs.k8s.io/
Apache License 2.0
395 stars 302 forks source link

Support managing multiple tagged security groups for a node #666

Open damdo opened 1 year ago

damdo commented 1 year ago

What would you like to be added: Add support for managing multiple tagged security groups for a node.

Why is this needed: At the moment we don't support finding and managing multiple tagged SecurityGroups per node (only one), and when we find more than one we error (ref code here).

As such if a node has 2 or more tagged SecurityGroups we would fail to modify them in order to allow inbound traffic from the LB SecurityGroup.

Is there anything preventing us to extend this functionality to more than one tagged SecurityGroup? If not, would it be possible to consider implementing it? Thanks

/kind feature

damdo commented 1 year ago

@cartermckinnon do you have any thoughts on this one? Thanks!

damdo commented 1 year ago

cc. @dims

dims commented 1 year ago

cc @kmala @oliviassss @M00nF1sh

kmala commented 1 year ago

I don't think there is any technical issue for having more than one tagged security groups as far as i can tell. This code was added long back when SG's were enabled which used to be a warning but latest got changed to error with the reason mentioned as comment for the function . So, i think its mostly security related than any limitation. Can you please describe your use case a bit more in detail on why/how the instance would have multiple SG with tags?

damdo commented 1 year ago

Sure @kmala. By default when we install Kubernetes we create a worker security group (SG1), which we then tag for CCM ownership. Then we let users create extra security groups (SG2,..., SGx) that they can attach to the worker instances without touching the default worker SG (SG1).

When one of those workers get attached to the Load Balancer, the LB security group rules are propagated by the CCM to the tagged Security Group attached to the worker instance.

Here though, in our case, we have multiple security groups, but since the CCM only considers one SG with the tag, the rule changes get propagated only to that (SG1), and not also to the other SGs attached (SG2,..., SGx).

This causes traffic issues.

kmala commented 1 year ago

/triage accepted

k8s-ci-robot commented 1 year ago

@kmala: The label triage/accepted cannot be applied. Only GitHub organization members can add the label.

In response to [this](https://github.com/kubernetes/cloud-provider-aws/issues/666#issuecomment-1762169642): >/triage accepted Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
kmala commented 1 year ago

Sounds good. I think we can add support for this.

cartermckinnon commented 1 year ago

/triage accepted

sounds legit to me!

damdo commented 1 year ago

Great! Thanks @kmala & @cartermckinnon How do you think it is best to proceed here to implement this?

k8s-triage-robot commented 1 week ago

This issue has not been updated in over 1 year, and should be re-triaged.

You can:

For more details on the triage process, see https://www.kubernetes.dev/docs/guide/issue-triage/

/remove-triage accepted

damdo commented 1 week ago

/triage accepted