search

kubernetes / cloud-provider-aws

Cloud provider for AWS
https://cloud-provider-aws.sigs.k8s.io/
Apache License 2.0
395 stars 302 forks source link

Prefix delegation based IPAM support in cc provider extension for IPv6 #608

Closed philsbln closed 6 months ago

philsbln commented 1 year ago

What would you like to be added:

We would like to integrate an IP address management (IPAM) controller with the AWS cloud-controller-manager to assign POD addresses based on prefix delegations. Our primary use-case are IPv6 only clusters, but the concept would also work with some limitations for IPv4.

The desired process for assigning IPv6 addresses to pods is as follows:

Why is this needed:

While adding IPv6 support to the gardener project, we strive to get globally unique IPv6 addresses across all our clusters. We would prefer to integrate the IPv6 IPAM functionality with the cloud provider's as much as possible as using provider managed IPv6 space also eliminates the need for NAT or routing hacks.

The functionality to use delegated prefixes as PodCIDR is already implemented in amazon-vpc-cni-k8s, which requires api keys to read/add prefix delegations to be present on the nodes. This implementation was reasonable for IPv4, where nodes needed dynamically add multiple prefix delegations in order to preserve precious address space. For IPv6, we only need a single prefix delegation and can add this one at the time we create the node, thus, eliminating the risk to expose API keys through a compromised node and the need to deploy the vpc-cni to the nodes.

/kind feature

k8s-ci-robot commented 1 year ago

This issue is currently awaiting triage.

If cloud-provider-aws contributors determine this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
olemarkus commented 1 year ago

This sounds identical to what kOps does, allowing IPv6 prefix support regardless of which CNI is used. We have been discussing if the kOps address controller should be moved to CCM before.

philsbln commented 1 year ago

This sounds identical to what kOps does, allowing IPv6 prefix support regardless of which CNI is used. We have been discussing if the kOps address controller should be moved to CCM before.

It's not surprising as kOps and Gardner both manage cluster lifecycle and orchestrate clusters across different cloud providers and I agree https://github.com/kubernetes/kops/blob/master/cmd/kops-controller/controllers/awsipam.go looks very much like what we would need to implement if it was not available through the provider extension…

DockToFuture commented 1 year ago

What was the reason to not move it to CCM? It would be a value proposition to a lot of people.

olemarkus commented 1 year ago

From my side, it's only the time it takes to do it. When it was implemented, it was simpler to get it into kops than into this project.

M00nF1sh commented 1 year ago

For my preference, i think CCM should just provide the bare minimal functionality to implement the kubernete's cloudProvider interface.

IP assignment better to be implemented as a standalone controller, which allows it to iterate independently than CCM, and allows users to supply different implementations. Is there any technically reasons what this should be in CCM than a standalone controller?

johngmyers commented 1 year ago

This is well within CCM's bailiwick. It's a straightforward, simple reconciliation loop copying any IPv6 prefix assignment from the cloud API to the Kubernetes Node object.

DockToFuture commented 1 year ago

Im my opinion this also belongs into the bailiwick of the aws cloud-controller-manger. Other cloud-controller-manager like gcp have the same understanding.

k8s-triage-robot commented 9 months ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot commented 8 months ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot commented 7 months ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

k8s-ci-robot commented 7 months ago

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to [this](https://github.com/kubernetes/cloud-provider-aws/issues/608#issuecomment-2028027163): >The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. > >This bot triages issues according to the following rules: >- After 90d of inactivity, `lifecycle/stale` is applied >- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied >- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed > >You can: >- Reopen this issue with `/reopen` >- Mark this issue as fresh with `/remove-lifecycle rotten` >- Offer to help out with [Issue Triage][1] > >Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). > >/close not-planned > >[1]: https://www.kubernetes.dev/docs/guide/issue-triage/ Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
DockToFuture commented 7 months ago

/reopen

k8s-ci-robot commented 7 months ago

@DockToFuture: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to [this](https://github.com/kubernetes/cloud-provider-aws/issues/608#issuecomment-2032029246): >/reopen Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
philsbln commented 7 months ago

/reopen

k8s-ci-robot commented 7 months ago

@philsbln: Reopened this issue.

In response to [this](https://github.com/kubernetes/cloud-provider-aws/issues/608#issuecomment-2037097093): >/reopen Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
k8s-triage-robot commented 6 months ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

k8s-ci-robot commented 6 months ago

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to [this](https://github.com/kubernetes/cloud-provider-aws/issues/608#issuecomment-2094155278): >The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. > >This bot triages issues according to the following rules: >- After 90d of inactivity, `lifecycle/stale` is applied >- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied >- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed > >You can: >- Reopen this issue with `/reopen` >- Mark this issue as fresh with `/remove-lifecycle rotten` >- Offer to help out with [Issue Triage][1] > >Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). > >/close not-planned > >[1]: https://www.kubernetes.dev/docs/guide/issue-triage/ Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.