search

kubernetes-sigs / cluster-api-provider-packet

Cluster API Provider Packet (now Equinix Metal)
https://deploy.equinix.com/labs/cluster-api-provider-packet/
Apache License 2.0
97 stars 40 forks source link

Implement CAPI IPAM Provider (and Equinix Metal VRF support) #679

Open displague opened 6 months ago

displague commented 6 months ago

User Story

Equinix Metal is a provider where the new CAPI IPAM features could be beneficial.

In Equinix Metal, Layer2 and Hybrid modes give users more control over the IPs and routing of their nodes. The VRF feature allows for IPAM blocks to be defined for additional routing benefits, through interconnection virtual circuits and gateways. IP assignments to individual nodes from the VRF block of IPs is currently unmanaged. The IPs are node / ARP managed. The Cluster API IPAM provider could be used to create VRF blocks and assign them to nodes (IP Claims) by creating IP reservation assignments in the Equinix Metal API. These assignments can include metadata (customdata, tags) that would map the Equinix Metal resources to the CAPI resources (IP Claims).

https://github.com/kubernetes-sigs/cluster-api/blob/main/docs/proposals/20220125-ipam-integration.md#motivation https://github.com/kubernetes-sigs/cluster-api-ipam-provider-in-cluster#cluster-api-ipam-provider-in-cluster

Detailed Description

Implement the IPAM provider in the CAPP provider. Follow the guidance set by the CAPV (and perhaps Metal3, others?) implementation.

https://github.com/kubernetes-sigs/cluster-api/blob/main/docs/proposals/20220125-ipam-integration.md#implementing-an-ipam-provider

https://github.com/kubernetes-sigs/cluster-api-provider-vsphere/blob/main/docs/node-ipam-demo.md

Anything else you would like to add:

/kind feature

k8s-triage-robot commented 3 months ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot commented 2 months ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot commented 1 month ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

k8s-ci-robot commented 1 month ago

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to [this](https://github.com/kubernetes-sigs/cluster-api-provider-packet/issues/679#issuecomment-2094302710): >The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. > >This bot triages issues according to the following rules: >- After 90d of inactivity, `lifecycle/stale` is applied >- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied >- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed > >You can: >- Reopen this issue with `/reopen` >- Mark this issue as fresh with `/remove-lifecycle rotten` >- Offer to help out with [Issue Triage][1] > >Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). > >/close not-planned > >[1]: https://www.kubernetes.dev/docs/guide/issue-triage/ Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
cprivitere commented 1 month ago

/reopen

k8s-ci-robot commented 1 month ago

@cprivitere: Reopened this issue.

In response to [this](https://github.com/kubernetes-sigs/cluster-api-provider-packet/issues/679#issuecomment-2094941977): >/reopen Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
cprivitere commented 1 month ago

/remove-lifecycle rotten

rahulii commented 1 month ago

hi @displague, here is my understanding of the feature: Once, we have integrated IPAM Provider with CAPP, here is how it would work (please correct me If I am wrong):

  1. User installs CAPP with IPAM Provider.

  2. User creates an IPPool Custom Resource with a specific IP subnet defined (eg: 10.10.10.0/24) from which the IPAM will allocate the IP addresses. For example:

    apiVersion: ipam.cluster.x-k8s.io/v1alpha1
kind: IPPool
metadata:
  name: some-pool
spec:
pools:
- subnet: 10.10.10.0/24
  start: 10.10.10.100
  end: 10.10.10.200

  3. The IPAM provider will use the EM API to create '10.10.10.0/24' IP address reservation in VRF. (how to know which VRF to add IP address reservations to ?) or will it create a new VRF altogether ??

  4. Once the IP Address reservation is created - IPAM can start allocating IPAddress on IPAddressClaim Request.

Please correct me if I have misunderstood anything (mainly point no.3 ) Note: This does not talks about how to actually implement IPAM. that will be done seperately.

displague commented 1 month ago

(how to know which VRF to add IP address reservations to ?) or will it create a new VRF altogether ??

With all of our IP options, IP Reservations, LoadBalancers (#740), VRF, I think it is going to be helpful to give the options for users to select an existing resource or (default) create a new one. There will also be a need to be ways to indicate how deletions work. Whether these options can be exposed as spec or annotations will depend on how the Kubernetes resource is defined, whether our controllers define that spec or whether they are defined upstream.

The machine in the cluster will need to join the VLAN (through the Equinix Metal APIs) where the VRF is attached. The IPAM/VRF request/definition will need to specify this somehow.

The local OS networking will also need to be configured, this may mean setting up IPs and it may mean attaching to VLANs, again at an OS level. If the IPAM Provider framework offers general-purpose tools like the CSI Attacher (which manages OS level mounting), I would hope that VLAN options would be exposed through that tool.