Azure / karpenter-provider-azure

AKS Karpenter Provider
Apache License 2.0
308 stars 47 forks source link

feat: Support for static AzureCNI without overlay networking via generating additional ip configurations #365

Open Bryce-Soghigian opened 1 month ago

Bryce-Soghigian commented 1 month ago

Fixes #367

Description This PR adds support for azure cni without overlay, as well as introduces some makefile goodness for creating clusters or other cni configurations.

Why Do We Need Secondary IP Configs For AZ CNI Without Overlay?

When a pod is created, the Azure CNI plugin allocates an IP address from the pool of secondary IP addresses configured on the NIC of the node where the pod is scheduled. The Azure CNI plugin manages the allocation and de-allocation of these IP addresses through the IP Address Manager (IPAM), ensuring each pod receives a unique IP address and tracking the usage of these addresses.

In this setup, pods are assigned IP addresses from the node's subnet, allowing for direct IP connectivity. This enables pods within the same virtual network to communicate without the need for Network Address Translation (NAT). The node's NIC routes traffic to the appropriate pod based on the assigned IP.

Flow

  1. Node NIC: Primary IP and multiple secondary IP addresses are assigned to the nic on node creation, and the nic is assigned to the node
  2. Pod Create: Pod requests an IP address on intialization
  3. Azure CNI Plugin: Assigns a secondary IP to the pod from the node’s NIC.
  4. Network Interface within Node: We use transparent mode, which doesn't change any properties of the eth0 interface on the nic. Azure CNI creates and adds host-side pod veth pair interfaces that are added to the host network.
  5. Pod to Pod Communication: Pods communicate using their assigned IPs directly within the virtual network. Pod to Pod communication is over layer 3 and L3 routing rules route the pod traffic.

Learn more about specifics here

How was this change tested?

What this PR does not include

Release Note

support added for static AzureCNI without overlay networking
coveralls commented 1 month ago

Pull Request Test Coverage Report for Build 9359164104

Details


Changes Missing Coverage Covered Lines Changed/Added Lines %
pkg/utils/cni.go 7 8 87.5%
pkg/providers/launchtemplate/launchtemplate.go 9 11 81.82%
<!-- Total: 102 105 97.14% -->
Totals Coverage Status
Change from base Build 9074172591: 0.003%
Covered Lines: 36334
Relevant Lines: 37160

💛 - Coveralls
Bryce-Soghigian commented 1 month ago

Still figuring out a bug in the make ci-non-test commands for new files its generating the boilerplate twice. But this should be ready for review @charliedmcb @tallaxes